Mailman 3 September 2026 - NumPy-Discussion

next NumPy New Contributors' Hour - Thursday, September 17th, 2026 at 10 pm UTC
by Inessa Pawson Oct. 12, 2026

Oct. 12, 2026

Our next New Contributors' Hour will be held TOMORROW, September 17th at 10 pm UTC. Stop by to ask questions, share your progress, or just to say hi. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/3f3otyyuTte3FU9y3QzsLg?both Join the meeting via Zoom: https://us06web.zoom.us/j/82563808729?pwd=ZFU3Z2dMcXBGb05YemRsaGE1OW5nQT09 -- Cheers, Inessa Inessa Pawson GitHub: inessapawson

1 1

next NumPy triage meeting - Wednesday, February 18th, 2026 at 18:00 UTC
by Inessa Pawson Oct. 11, 2026

Oct. 11, 2026

Our next triage meeting will be held this Wednesday, February 18th at 18:00 (6 pm) UTC. This is a meeting where we synchronously triage prioritized PRs and issues. Everyone is welcome to attend and contribute to a conversation. Join us via Zoom: https://numfocus-org.zoom.us/j/82096749952?pwd=MW9oUmtKQ1c3a2gydGk1RTdYUUVX… If there are PRs or issues you’d like us to review, please add GitHub links to them to the meeting agenda: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg -- Cheers, Inessa Inessa Pawson GitHub: inessapawson

1 16

Behaviour of Logical Operands "|" and "&" with Numpy.Array
by zhong.ann＠gmail.com Sept. 30, 2026

Sept. 30, 2026

Hi, I would like to report weird behaviours of the logical operands "|" and "&" when interacting with Numpy.Arrays of integer and boolean elements. I have observed that if we put "|" or "&" between a Numpy.Array of integers and a Numpy.Array of booleans, "|" would increase the non-zero even integers by 1 in the returned output, while "&" would treat the non-zero even integers as 0 and return 0 for them. For example: numpy.array([-2, -1, 0, 1, 2, 3, 4]) | numpy.array([True]) would generate result as array([-1, -1, 1, 1, 3, 3, 5]). numpy.array([-2, -1, 0, 1, 2, 3, 4]) & numpy.array([True]) would generate result as array([0, 1, 0, 1, 0, 1, 0]) . According to the numpy user guide, the operands "|" and "&" are equivalent to logical_or() and logical_and() respectively, but in fact logical_or() and logical_and() return an array of boolean values when given both an array of integer and an array of boolean as input. Note that the logical operands "|" and "&" are not supported by python built-in array.array to work between an array of integer and an array of boolean. Could you please advise if these logical operands should be supported to operate on an array of integer and an array of boolean? If they should, what would be the expected behaviours? Thank you.

3 4

Proposal for new DType API slot
by Nathan Sept. 27, 2026

Sept. 27, 2026

Hi all, I went down a bit of a rabbit hole with the aid of an AI model today related to my ByteStringDType and ongoing work to improve StringDType. It led me to conclude that there is a missing feature in the DType API. I'd appreciate it if I could get some feedback about whether I'm on the right track. ## Problem Statement Consider this example with np.nan and StringDType: ```python import numpy as np from numpy.dtypes import StringDType text = np.array(["a", "b"], dtype=StringDType(na_object=np.nan)) text[1] = np.nan # stores a missing value text.tolist() # ['a', nan] np.where([True, False], text, np.nan) # raises DTypePromotionError ``` Here, `np.where` treats `np.nan` as a float, so it gets promoted as such, and we end up raising an exception *despite* the `na_object` being set to `np.nan`. Other functions lose dtype information or string contents when converting Python values independently: ```python words = np.array(["a", "longer"], dtype=StringDType()) np.vectorize(lambda s: s)(words).dtype # dtype('<U6') np.apply_along_axis(lambda row: row[0], 1, words[:, None]).tolist() # ['a', 'l'] nul_text = np.array(["x", "x\0"], dtype=StringDType()) np.isin(nul_text, ["x\0"]).tolist() # [True, False] ``` Related problems affect `select`, `append`, `concatenate`, and set operations. Let's consider some hypothetical user defined DTypes: ```python rank = CategoricalDType(["low", "medium", "high"], ordered=True) levels = np.array(["low", "high"], dtype=rank) levels[0] # 'low', a Python str np.sort(levels).tolist() # ['low', 'high']: category order levels[0] = "medium" # accepted levels[0] = "urgent" # error: unknown category ``` ```python metres = UnitDType("m") lengths = np.array([1.0, 2.0], dtype=metres) lengths[0] = 3.0 # stores three metres lengths.astype(UnitDType("cm")) # values [300.0, 200.0] in centimetres lengths.astype(UnitDType("s")) # error: incompatible units ``` Both the unit and categorical DTypes have a similar issue to the one StringDType has above: plain python scalars are compatible but rejected because promotion happens without considering this possibility. For example, with `np.append` (via `concatenate`) ```python np.append(levels, ["medium"]) # error: cannot promote rank with Unicode np.append(lengths, [3.0]) # error: cannot promote metres with float64 ``` My reading of NEP 41-43 is that these issues came up but were implicitly deferred. For example, the discussion about categorical dtypes losing their dtype when converted back to arrays in NEP 42. I think this mechanism might be able to help these deferred cases. ## Proposal I'd like to propose a new DType API hook `NPY_DT_discover_descr_with_context` and typedef: ```c int discover_descr_with_context( npy_intp ndescrs, PyArray_Descr *const descrs[], PyObject *value, NPY_DTYPE_CONTEXT context, PyArray_Descr **out); ``` For most common things there will only be one input descriptor: ```python a = np.array(["hello"], dtype=StringDType(na_object=np.nan)) np.where([False], a, np.nan) ``` In this example the one input descriptor is a.dtype. It's also possible to call `np.vectorize` with an arbitrary number of input arrays, so this takes an arbitrary number of inputs, but the hook only fires if all inputs have the same dtype. The `value` is a Python object that could be a possible scalar for the dtype instance. In the `np.where` example above, the `value` is `np.nan`. The NPY_DTYPE_CONTEXT enum tells this function the kind of operation that is being handled and then `out` is filled in with the appropriate descriptor for `value`. You can also return 0 to indicate that you have no special instructions. The three types of `context` argument I'd like to propose are: - **OPERAND:** convert one input, then promote it with the other inputs. - **SEQUENCE_ELEMENT:** require every scalar element to be accepted by the same DType, combine their settings, then convert the sequence. - **RESULT:** choose the dtype for a value returned by the user's function. NumPy calls this hook through the private `_array_converter` methods `as_arrays(with_context=True)` and `result_type_hint(value)`, retaining original values until conversion. StringDType can then recognize strings and its configured missing values before conversion. On the proposed branch, the earlier examples produce: ```python np.where([True, False], text, np.nan).tolist() # ['a', nan] np.vectorize(lambda s: s)(words).dtype # StringDType() np.apply_along_axis(lambda row: row[0], 1, words[:, None]).tolist() # ['a', 'longer'] np.isin(nul_text, ["x\0"]).tolist() # [False, True] ``` The hypothetical dtypes could likewise make `append` follow their assignment rules while continuing to reject implicit conversion of typed arrays: ```python # With hooks for the hypothetical dtypes: np.append(levels, ["medium"]) # ['medium', 'high', 'medium'], dtype=rank np.append(lengths, [3.0]) # values [3.0, 2.0, 3.0] in metres ``` I've coded (with substantial AI assistance) an [experimental implementation]( https://github.com/numpy/numpy/compare/main...ngoldbaum:numpy:context-dtype…) that includes StringDType support.

3 3

Simplifying releases
by Charles R Harris Sept. 27, 2026

Sept. 27, 2026

Hi All, I am thinking we should simplify our releases on GitHub. Currently I post release notes, translated from rst to gpm, and upload a source file, a changelog, and the release notes in rst. I am thinking that all we really need to do is post the release notes, with a footer warning not to use the GitHub generated release products, and an admonition to download from PyPI. That leaves no security holes for the source, and the changelog can be accessed on GitHub itself. Thoughts? Chuck

3 2

NEP 58: A variable-width bytes DType
by Nathan Sept. 22, 2026

Sept. 22, 2026

Hi all, I just opened a PR to add NEP 58, which proposes a new variable-width dtype, corresponding to the python bytes type. See https://github.com/numpy/numpy/pull/32433. Per NEP 0, I've copied the NEP content up through "Usage and Impact". Please keep substantive comments about the NEP as a whole in this thread rather than in the github PR for the NEP text. However, please direct specific comments about the text to the PR. Thanks all for your ideas and input on this, -Nathan ===================================== NEP 58 — A variable-width bytes DType ===================================== :Author: Nathan Goldbaum :Status: Draft :Type: Standards Track :Created: 2026-08-25 Abstract -------- I propose ``ByteStringDType``, a variable-width bytes data type: the bytes sibling of ``StringDType`` (:ref:`NEP 55 <NEP55>`). It reuses StringDType's arena-backed storage, allocator, and missing-data machinery while * storing and returning Python :class:`bytes`, * supporting embedded and trailing NUL bytes by construction, and * exposing only operations meaningful on raw bytes. Text and bytes never promote or cast implicitly. The only conversion between ``StringDType`` and ``ByteStringDType`` is via the ``np.strings.encode``/``np.strings.decode`` pair. These gain a C UFunc implementation, along with the ability to add codec-aware variable-width loops mirroring :meth:`str.encode` and :meth:`bytes.decode`. A working prototype accompanies this NEP as its reference implementation. Motivation and scope -------------------- NumPy's only bytes data type is the fixed-width ``S`` dtype (``np.dtypes.BytesDType``, scalar ``np.bytes_``). Because ``S`` is null-padded, it cannot represent trailing NUL bytes:: >>> np.array([b"x\x00"])[0] b'x' This makes ``S`` unsuitable for generic byte streams such as binary record formats, encoded blobs, and network data. The truncation cannot be fixed in place: existing code relies on it, so reports have been closed as not planned since 2011 (NumPy `issue #2414 <https://github.com/numpy/numpy/issues/2414>`__; more recently `issue #25268 <https://github.com/numpy/numpy/issues/25268>`__, which loses the final byte of a SHA-256 digest). :ref:`NEP 55 <NEP55>` called this out, explicitly ruled a bytes/arbitrary-encoding dtype out of its scope, and listed an improved binary dtype as complementary future work. NumPy `issue #27701 <https://github.com/numpy/numpy/issues/27701>`__ is the open request for a StringDType equivalent for ``bytes``. This NEP proposes that dtype. Many downstream libraries fall back to object arrays of ``bytes`` wherever they carry variable-width binary data, giving up NumPy's flat memory layout and loop machinery. PyArrow materializes every Arrow binary column as an `object array of boxed bytes objects <https://github.com/apache/arrow/blob/118892700b95fdfa9a6b3e482a6e5399563f5d…>`_, and the reverse conversion from ``S`` has to guess each element's length with ``strnlen``, `truncating at the first NUL <https://github.com/apache/arrow/blob/118892700b95fdfa9a6b3e482a6e5399563f5d…>`_. The h5py library reads HDF5 variable-length byte strings `as object arrays <https://github.com/h5py/h5py/blob/821e503405b5e26a1333b28f2b6418d1a2f8c88a/…>`_. Zarr defines a first-class ``variable_length_bytes`` data type and `stores it in object arrays <https://github.com/zarr-developers/zarr-python/blob/20ba31e3e1142fae83b178d…>`_ for the same reason. Astropy holds FITS variable-length binary columns in object arrays and cannot round-trip fixed-width character columns without rewriting their padding (`astropy#11341 <https://github.com/astropy/astropy/issues/11341>`__). pandas inherits the ``S`` truncation for ``bytes`` columns (`pandas#58205 <https://github.com/pandas-dev/pandas/issues/58205>`__). In scope: * A variable-width, NUL-transparent bytes DType with the same operation *set* as fixed-width ``S`` (ASCII case folding and predicates, byte-indexed search/slice), the same missing-data support as StringDType, and casts to/from ``S``, void, and bool. * Explicit, codec-aware ``encode`` and ``decode`` ufuncs as the only text-to-bytes path for the variable-width pair. * As a structural side effect, StringDType's UTF-8 assumptions are identified and confined to a small encoding-specific surface of the implementation. Out of scope: * Changing how Python ``bytes`` values are inferred (``np.array([b"x"])`` stays fixed-width ``S``, as NEP 55 kept ``str`` inference at ``U``). * Exposing other encodings (latin-1, utf-16) as array dtypes. The encoding-specific surface identified here is a starting point for an encoding-parameterized StringDType, but that is a new user-facing semantic that would need its own proposal. Usage and impact ---------------- Because it uses the same representation and arena-backed storage as ``StringDType``, the new ``ByteStringDType`` supports embedded and trailing NUL bytes automatically: .. code-block:: python >>> import numpy as np >>> from numpy.dtypes import ByteStringDType >>> a = np.array([b"x\x00", b"a\x00b", b"\xff\xfe"], dtype=ByteStringDType()) >>> a[0] # trailing NULs survive b'x\x00' >>> a[1] # embedded NULs too b'a\x00b' >>> np.strings.str_len(a) # lengths are in bytes, length-explicit array([2, 3, 2]) >>> np.strings.find(a, b"\x00") array([ 1, 1, -1]) This dtype does not support the ``coerce`` argument that ``StringDType`` supports, so data that is not bytes will be rejected by ``np.array()``: .. code-block:: python >>> np.array(["text"], dtype=ByteStringDType()) Traceback (most recent call last): ... TypeError: ByteStringDType only allows bytes data, got an instance of 'str'; convert text to bytes explicitly with str.encode(encoding) Converting between ``StringDType`` and ``ByteStringDType`` happens through ``np.strings.encode`` and ``np.strings.decode``. While the ``encode`` default transitions (see :ref:`backward_compatibility`), the ByteStringDType result is requested explicitly: >>> s = np.array(["héllo"], dtype=np.dtypes.StringDType()) >>> b = np.strings.encode(s, "utf-8", dtype=ByteStringDType()) >>> np.strings.decode(b, "utf-8") array(['héllo'], dtype=StringDType()) .. _backward_compatibility: Backward compatibility ---------------------- There is only one major backward compatibility concern: dealing with ``np.strings.encode``. The function already exists and supports ``StringDType``, but sub-optimally in a manner that cannot perserve trailing NUL bytes. This presents some awkward backward compatibility concerns for this proposal. In the prototype branch, ``np.strings.encode`` emits a ``DeprecationWarning`` for StringDType input when the new ``dtype=`` argument is unspecified. Until the default flips to ByteStringDType in a later release, its behavior is otherwise unchanged: fixed-width ``S`` result, full codec and error-mode support, and 0-d arrays for 0-d input. See :ref:`encoding_decoding` for more detail and :ref:`open_questions` for whether this deprecation should happen.

3 7

next NumPy community meeting - Wednesday, July 29th, 2026 at 18:00 UTC
by Inessa Pawson Sept. 20, 2026

Sept. 20, 2026

The next NumPy community meeting will be held this Wednesday, July 29th at 18:00 (6 pm) UTC. Join us via Zoom: https://numfocus-org.zoom.us/j/83278611437?pwd=ekhoLzlHRjdWc0NOY2FQM0NPemdk… Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both For the notes from the previous meetings, visit: https://github.com/numpy/archive/tree/main/community_meetings -- Cheers, Inessa Inessa Pawson GitHub: inessapawson

2 5

next NumPy Optimization Team meeting - Monday, August 10th, 2026 at 17:00 UTC
by Inessa Pawson Sept. 19, 2026

Sept. 19, 2026

Hi, all! The next NumPy Optimization Team meeting will be held on Monday, August 10th at 17:00 (5 pm) UTC. Join us via Zoom: https://numfocus-org.zoom.us/j/81261288210?pwd=iwV99tGSjR61RTGEERKM4QKxe46g… Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/dVdSlQ0TThWkOk0OkmGsmw?both For the notes from the previous meetings, visit: https://github.com/numpy/archive/tree/main/optim_team_meetings -- Cheers, Inessa Inessa Pawson GitHub: inessapawson

1 2

Welcome Iason Kryommydas as a new maintainer
by Nathan Sept. 18, 2026

Sept. 18, 2026

Hi all, It's my pleasure to welcome Iason (@irkommyd on GitHub) as a new maintainer! You can read about the work he did over the summer to improve NumPy's ufunc infrastructure on the Quansight blog: https://labs.quansight.org/blog/teaching-numpys-ufuncs-new-tricks Iason has been participating in code reviews regularly and we want to encourage more of that by giving him a commit bit. Thanks for all you've done Iason and I'm looking forward to working with you more in the future! -Nathan

7 6

ANN: numpy-financial 1.1.0
by Inessa Pawson Sept. 17, 2026

Sept. 17, 2026

Hi, all! I'm pleased to announce the release of numpy-financial v1.1.0. Many thanks to Stéfan van der Walt for his hard work on getting this release across the finish line and keeping the project moving forward. ************************** Sources can be found at ************************** https://pypi.org/project/numpy-financial/ https://github.com/numpy/numpy-financial/releases/tag/v1.1.0 ************************** Highlights of this release ************************** ENH: Use Newton's method to calculate IRR (#47). ENH: Allow npv calculation to be broadcastable (#54). Avoid redundant computations in IRR calculation (#60). ENH: NPV: Support calculation for vectors of rates and cashflows (#96). REL: Update minimum required versions (#102). ENH: npv: Rework npv function to mimic broadcasting (#108). Updated irr function [issue 98] (#99). ENH: nper: broadcast rework with Cython (#118). ENH: nper: Perform C division where appropriate (#120). ENH: mirr: Mimic broadcasting (#125). TYP: Inline typing annotations (#136). ************************** Contributors ************************** **17 authors contributed to this release (alphabetically):** @AlexMGTNO @Eugenia-Mazur @jlopezpena Daniel McCloy (@drammock) Jamie Cook (@jamiecook) Jasper Lee (@yatshunlee) Joren Hammudoglu (@jorenham) Josh Wilson (@person142) Kai Striega (@Kai-Striega) Maharshi Basu (@MashyBasker) Mandeep Singh (@mandeep-singh-sndk) Melissa Weber Mendonça (@melissawm) Mika (@tal66) SeanZ (@seanzian2093) Sebastian Berg (@seberg) Stefan van der Walt (@stefanv) Warren Weckesser (@WarrenWeckesser) **17 reviewers contributed to this release (alphabetically):** @101AlexMartin @AlexMGTNO @Eugenia-Mazur Daniel McCloy (@drammock) Inessa Pawson (@InessaPawson) Jamie Cook (@jamiecook) Jasper Lee (@yatshunlee) Joren Hammudoglu (@jorenham) Josh Wilson (@person142) Kai Striega (@Kai-Striega) Maharshi Basu (@MashyBasker) Melissa Weber Mendonça (@melissawm) Mika (@tal66) Ralf Gommers (@rgommers) SeanZ (@seanzian2093) Stefan van der Walt (@stefanv) Warren Weckesser (@WarrenWeckesser) **************************************************** For full release notes, visit: https://github.com/numpy/numpy-financial/releases/tag/v1.1.0 -- Inessa Inessa Pawson GitHub: inessapawson

1 0