Mailman 3 August 2023 - NumPy-Discussion

Add to NumPy a function to compute cumulative sums from 0.
by john.dawson＠camlingroup.com March 20, 2024

March 20, 2024

`cumsum` computes the sum of the first k summands for every k from 1. Judging by my experience, it is more often useful to compute the sum of the first k summands for every k from 0, as `cumsum`'s behaviour leads to fencepost-like problems. https://en.wikipedia.org/wiki/Off-by-one_error#Fencepost_error For example, `cumsum` is not the inverse of `diff`. I propose adding a function to NumPy to compute cumulative sums beginning with 0, that is, an inverse of `diff`. It might be called `cumsum0`. The following code is probably not the best way to implement it, but it illustrates the desired behaviour. ``` def cumsum0(a, axis=None, dtype=None, out=None): """ Return the cumulative sum of the elements along a given axis, beginning with 0. cumsum0 does the same as cumsum except that cumsum computes the sum of the first k summands for every k from 1 and cumsum, from 0. Parameters ---------- a : array_like Input array. axis : int, optional Axis along which the cumulative sum is computed. The default (None) is to compute the cumulative sum over the flattened array. dtype : dtype, optional Type of the returned array and of the accumulator in which the elements are summed. If `dtype` is not specified, it defaults to the dtype of `a`, unless `a` has an integer dtype with a precision less than that of the default platform integer. In that case, the default platform integer is used. out : ndarray, optional Alternative output array in which to place the result. It must have the same shape and buffer length as the expected output but the type will be cast if necessary. See :ref:`ufuncs-output-type` for more details. Returns ------- cumsum0_along_axis : ndarray. A new array holding the result is returned unless `out` is specified, in which case a reference to `out` is returned. If `axis` is not None the result has the same shape as `a` except along `axis`, where the dimension is smaller by 1. See Also -------- cumsum : Cumulatively sum array elements, beginning with the first. sum : Sum array elements. trapz : Integration of array values using the composite trapezoidal rule. diff : Calculate the n-th discrete difference along given axis. Notes ----- Arithmetic is modular when using integer types, and no error is raised on overflow. ``cumsum0(a)[-1]`` may not be equal to ``sum(a)`` for floating-point values since ``sum`` may use a pairwise summation routine, reducing the roundoff-error. See `sum` for more information. Examples -------- >>> a = np.array([[1, 2, 3], [4, 5, 6]]) >>> a array([[1, 2, 3], [4, 5, 6]]) >>> np.cumsum0(a) array([ 0, 1, 3, 6, 10, 15, 21]) >>> np.cumsum0(a, dtype=float) # specifies type of output value(s) array([ 0., 1., 3., 6., 10., 15., 21.]) >>> np.cumsum0(a, axis=0) # sum over rows for each of the 3 columns array([[0, 0, 0], [1, 2, 3], [5, 7, 9]]) >>> np.cumsum0(a, axis=1) # sum over columns for each of the 2 rows array([[ 0, 1, 3, 6], [ 0, 4, 9, 15]]) ``cumsum(b)[-1]`` may not be equal to ``sum(b)`` >>> b = np.array([1, 2e-9, 3e-9] * 1000000) >>> np.cumsum0(b)[-1] 1000000.0050045159 >>> b.sum() 1000000.0050000029 """ empty = a.take([], axis=axis) zero = empty.sum(axis, dtype=dtype, keepdims=True) later_cumsum = a.cumsum(axis, dtype=dtype) return concatenate([zero, later_cumsum], axis=axis, dtype=dtype, out=out) ```

14 25

next NumPy triage meeting - now starting at 5 pm UTC
by Inessa Pawson March 17, 2024

March 17, 2024

The next NumPy triage meeting will be held this Wednesday, January 11th at 5 pm UTC. (Please note the new time!) This is a meeting where we synchronously triage prioritized PRs and issues. Join us via Zoom: https://numfocus-org.zoom.us/j/82096749952?pwd=MW9oUmtKQ1c3a2gydGk1RTdYUUVX… Everyone is welcome to attend and contribute to a conversation. Please notify us of issues or PRs that you’d like to have reviewed by adding a GitHub link to them in the meeting agenda: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg. -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

1 30

NEP 55 - Add a UTF-8 Variable-Width String DType to NumPy
by Nathan Feb. 12, 2024

Feb. 12, 2024

Hello all, I just opened a pull request to add NEP 55, see https://github.com/numpy/numpy/pull/24483. Per NEP 0, I've copied everything up to the "detailed description" section below. I'm looking forward to your feedback on this. -Nathan Goldbaum ========================================================= NEP 55 — Add a UTF-8 Variable-Width String DType to NumPy ========================================================= :Author: Nathan Goldbaum <ngoldbaum(a)quansight.com> :Status: Draft :Type: Standards Track :Created: 2023-06-29 Abstract -------- We propose adding a new string data type to NumPy where each item in the array is an arbitrary length UTF-8 encoded string. This will enable performance, memory usage, and usability improvements for NumPy users, including: * Memory savings for workflows that currently use fixed-width strings and store primarily ASCII data or a mix of short and long strings in a single NumPy array. * Downstream libraries and users will be able to move away from object arrays currently used as a substitute for variable-length string arrays, unlocking performance improvements by avoiding passes over the data outside of NumPy. * A more intuitive user-facing API for working with arrays of Python strings, without a need to think about the in-memory array representation. Motivation and Scope -------------------- First, we will describe how the current state of support for string or string-like data in NumPy arose. Next, we will summarize the last major previous discussion about this topic. Finally, we will describe the scope of the proposed changes to NumPy as well as changes that are explicitly out of scope of this proposal. History of String Support in Numpy ********************************** Support in NumPy for textual data evolved organically in response to early user needs and then changes in the Python ecosystem. Support for strings was added to numpy to support users of the NumArray ``chararray`` type. Remnants of this are still visible in the NumPy API: string-related functionality lives in ``np.char``, to support the obsolete ``np.char.chararray`` class, deprecated since NumPy 1.4 in favor of string DTypes. NumPy's ``bytes_`` DType was originally used to represent the Python 2 ``str `` type before Python 3 support was added to NumPy. The bytes DType makes the most sense when it is used to represent Python 2 strings or other null-terminated byte sequences. However, ignoring data after the first null character means the ``bytes_`` DType is only suitable for bytestreams that do not contain nulls, so it is a poor match for generic bytestreams. The ``unicode`` DType was added to support the Python 2 ``unicode`` type. It stores data in 32-bit UCS-4 codepoints (e.g. a UTF-32 encoding), which makes for a straightforward implementation, but is inefficient for storing text that can be represented well using a one-byte ASCII or Latin-1 encoding. This was not a problem in Python 2, where ASCII or mostly-ASCII text could use the Python 2 ``str`` DType (the current ``bytes_`` DType). With the arrival of Python 3 support in NumPy, the string DTypes were largely left alone due to backward compatibility concerns, although the unicode DType became the default DType for ``str`` data and the old ``string`` DType was renamed the ``bytes_`` DType. This change left NumPy with the sub-optimal situation of shipping a data type originally intended for null-terminated bytestrings as the data type for *all* python ``bytes`` data, and a default string type with an in-memory representation that consumes four times as much memory as needed for ASCII or mostly-ASCII data. Problems with Fixed-Width Strings ********************************* Both existing string DTypes represent fixed-width sequences, allowing storage of the string data in the array buffer. This avoids adding out-of-band storage to NumPy, however, it makes for an awkward user interface. In particular, the maximum string size must be inferred by NumPy or estimated by the user before loading the data into a NumPy array or selecting an output DType for string operations. In the worst case, this requires an expensive pass over the full dataset to calculate the maximum length of an array element. It also wastes memory when array elements have varying lengths. Pathological cases where an array stores many short strings and a few very long strings are particularly bad for wasting memory. Downstream usage of string data in NumPy arrays has proven out the need for a variable-width string data type. In practice, most downstream users employ ``object`` arrays for this purpose. In particular, ``pandas`` has explicitly deprecated support for NumPy fixed-width strings, coerces NumPy fixed-width string arrays to ``object`` arrays, and in the future may switch to only supporting string data via ``PyArrow``, which has native support for UTF-8 encoded variable-width string arrays [1]_. This is unfortunate, since `` object`` arrays have no type guarantees, necessitating expensive sanitization passes and operations using object arrays cannot release the GIL. Previous Discussions -------------------- The project last discussed this topic in depth in 2017, when Julian Taylor proposed a fixed-width text data type parameterized by an encoding [2]_. This started a wide-ranging discussion about pain points for working with string data in NumPy and possible ways forward. In the end, the discussion identified two use-cases that the current support for strings does a poor job of handling: * Loading or memory-mapping scientific datasets with unknown encoding, * Working with string data in a manner that allows transparent conversion between NumPy arrays and Python strings, including support for missing strings. As a result of this discussion, improving support for string data was added to the NumPy project roadmap [3]_, with an explicit call-out to add a DType better suited to memory-mapping bytes with any or no encoding, and a variable-width string DType that supports missing data to replace usages of object string arrays. Proposed work ------------- This NEP proposes adding ``StringDType``, a DType that stores variable-width heap-allocated strings in Numpy arrays, to replace downstream usages of the ``object`` DType for string data. This work will heavily leverage recent improvements in NumPy to improve support for user-defined DTypes, so we will also necessarily be working on the data type internals in NumPy. In particular, we propose to: * Add a new variable-length string DType to NumPy, targeting NumPy 2.0. * Work out issues related to adding a DType implemented using the experimental DType API to NumPy itself. * Support for a user-provided missing data sentinel. * A cleanup of ``np.char``, with the ufunc-like functions moved to a new namespace for functions and types related to string support. * An update to the ``npy`` and ``npz`` file formats to allow storage of arbitrary-length sidecar data. The following is out of scope for this work: * Changing DType inference for string data. * Adding a DType for memory-mapping text in unknown encodings or a DType that attempts to fix issues with the ``bytes_`` DType. * Fully agreeing on the semantics of a missing data sentinels or adding a missing data sentinel to NumPy itself. * Implement fast ufuncs or SIMD optimizations for string operations. While we're explicitly ruling out implementing these items as part of this work, adding a new string DType helps set up future work that does implement some of these items. If implemented this NEP will make it easier to add a new fixed-width text DType in the future by moving string operations into a long-term supported namespace. We are also proposing a memory layout that should be amenable to writing fast ufuncs and SIMD optimization in some cases, increasing the payoff for writing string operations as SIMD-optimized ufuncs in the future. While we are not proposing adding a missing data sentinel to NumPy, we are proposing adding support for an optional, user-provided missing data sentinel, so this does move NumPy a little closer to officially supporting missing data. We are attempting to avoid resolving the disagreement described in :ref:`NEP 26<NEP26>` and this proposal does not require or preclude adding a missing data sentinel or bitflag-based missing data support in the future. Usage and Impact ---------------- The DType is intended as a drop-in replacement for object string arrays. This means that we intend to support as many downstream usages of object string arrays as possible, including all supported NumPy functionality. Pandas is the obvious first user, and substantial work has already occurred to add support in a fork of Pandas. ``scikit-learn`` also uses object string arrays and will be able to migrate to a DType with guarantees that the arrays contains only strings. Both h5py [4]_ and PyTables [5]_ will be able to add first-class support for variable-width UTF-8 encoded string datasets in HDF5. String data are heavily used in machine-learning workflows and downstream machine learning libraries will be able to leverage this new DType. Users who wish to load string data into NumPy and leverage NumPy features like fancy advanced indexing will have a natural choice that offers substantial memory savings over fixed-width unicode strings and better validation guarantees and overall integration with NumPy than object string arrays. Moving to a first-class string DType also removes the need to acquire the GIL during string operations, unlocking future optimizations that are impossible with object string arrays. Performance *********** Here we briefly describe preliminary performance measurements of the prototype version of ``StringDType`` we have implemented outside of NumPy using the experimental DType API. All benchmarks in this section were performed on a Dell XPS 13 9380 running Ubuntu 22.04 and Python 3.11.3 compiled using pyenv. NumPy, Pandas, and the ``StringDType`` prototype were all compiled with meson release builds. Currently, the ``StringDType`` prototype has comparable performance with object arrays and fixed-width string arrays. One exception is array creation from python strings, performance is somewhat slower than object arrays and comparable to fixed-width unicode arrays:: In [1]: from stringdtype import StringDType In [2]: import numpy as np In [3]: data = [str(i) * 10 for i in range(100_000)] In [4]: %timeit arr_object = np.array(data, dtype=object) 3.55 ms ± 51.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) In [5]: %timeit arr_stringdtype = np.array(data, dtype=StringDType()) 12.9 ms ± 277 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) In [6]: %timeit arr_strdtype = np.array(data, dtype=str) 11.7 ms ± 150 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) In this example, object DTypes are substantially faster because the objects in the ``data`` list can be directly interned in the array, while ``StrDType`` and ``StringDType`` need to copy the string data and ``StringDType`` needs to convert the data to UTF-8 and perform additional heap allocations outside the array buffer. In the future, if Python moves to a UTF-8 internal representation for strings, the string loading performance of ``StringDType`` should improve. String operations have similar performance:: In [7]: %timeit np.array([s.capitalize() for s in data], dtype=object) 30.2 ms ± 109 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) In [8]: %timeit np.char.capitalize(arr_stringdtype) 38.5 ms ± 3.01 ms per loop (mean ± std. dev. of 7 runs, 10 loops each) In [9]: %timeit np.char.capitalize(arr_strdtype) 46.4 ms ± 1.32 ms per loop (mean ± std. dev. of 7 runs, 10 loops each) The poor performance here is a reflection of the slow iterator-based implementation of operations in ``np.char``. If we were to rewrite these operations as ufuncs, we could unlock substantial performance improvements. Using the example of the ``add`` ufunc, which we have implemented for the ``StringDType`` prototype:: In [10]: %timeit arr_object + arr_object 10 ms ± 308 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) In [11]: %timeit arr_stringdtype + arr_stringdtype 5.91 ms ± 18.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) In [12]: %timeit np.char.add(arr_strdtype, arr_strdtype) 65.9 ms ± 1.3 ms per loop (mean ± std. dev. of 7 runs, 10 loops each) As described below, we have already updated a fork of Pandas to use a prototype version of ``StringDType``. This demonstrates the performance improvements available when data are already loaded into a NumPy array and are passed to a third-party library. Currently Pandas attempts to coerce all ``str`` data to ``object`` DType by default, and has to check and sanitize existing ``object `` arrays that are passed in. This requires a copy or pass over the data made unnecessary by first-class support for variable-width strings in both NumPy and Pandas:: In [13]: import pandas as pd In [14]: %timeit pd.Series(arr_stringdtype) 20.9 µs ± 341 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) In [15]: %timeit pd.Series(arr_object) 1.08 ms ± 23.4 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each) We have also implemented a Pandas extension DType that uses ``StringDType`` under the hood, which is also substantially faster for creating Pandas data structures than the existing Pandas string DType that uses ``object`` arrays:: In [16]: %timeit pd.Series(arr_stringdtype, dtype='string[numpy]') 54.7 µs ± 1.38 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each) In [17]: %timeit pd.Series(arr_object, dtype='string[python]') 1.39 ms ± 1.16 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each) Backward compatibility ---------------------- We are not proposing a change to DType inference for python strings and do not expect to see any impacts on existing usages of NumPy, besides warnings or errors related to new deprecations or expiring deprecations in ``np.char``.

6 14

next NumPy Newcomers' Hour - 12 pm UTC
by Inessa Pawson Oct. 31, 2023

Oct. 31, 2023

Our next Newcomers' Hour will be held tomorrow, December 1st, at 12 pm UTC. Stop by to ask questions or just to say hi. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/3f3otyyuTte3FU9y3QzsLg?both Join the meeting via Zoom: https://us06web.zoom.us/j/82563808729?pwd=ZFU3Z2dMcXBGb05YemRsaGE1OW5nQT09 -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

2 5

next NumPy community meeting
by Inessa Pawson Oct. 22, 2023

Oct. 22, 2023

The next NumPy community meeting will be held this Wednesday, August 17th at 18:00 (6 pm) UTC. Join us via Zoom: https://us06web.zoom.us/j/83278611437?pwd=ekhoLzlHRjdWc0NOY2FQM0NPemdkZz09 Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both For the notes from the previous meeting, visit: https://github.com/numpy/archive/blob/main/community_meetings/community-202… Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ Twitter: @inessapawson

1 30

Next Documentation team meeting
by Mukulika Pahari Sept. 11, 2023

Sept. 11, 2023

Hi all! Our next Documentation Team meeting will happen on *Monday, June 5* at ***4PM UTC***. We now alternate the meeting times to be a bit more inclusive. This means that we'll have a meeting at 12pm UTC every 28 days, and a meeting at 4pm UTC every 28 days. All are welcome - you don't need to already be a contributor to join. If you have questions or are curious about what we're doing, we'll be happy to meet you! If you wish to join on Zoom, use this (updated) link: https://numfocus-org.zoom.us/j/85016474448?pwd=TWEvaWJ1SklyVEpwNXUrcHV1YmFJQ... Here's the permanent hackmd document with the meeting notes (still being updated): https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg Hope to see you around! * You can also visit https://scientific-python.org/calendars to add the NumPy community calendar as an .ics file to your preferred calendar manager. * Best wishes, Mukulika

1 5

Dealing with static local variables in Numpy
by Nicolas Holzschuch Sept. 1, 2023

Sept. 1, 2023

Hello, This is my first post to this group; I'd like to start by expressing my appreciation for the amazing work in developing and maintaining Numpy. I have a question. Numpy has quite a lot of static local variables (variables defined as static inside a function, like this (core/src/multiarraymodule.c, line 4483): if (raise_exceptions) { static PyObject *too_hard_cls = NULL; /* ... */ } I understand that these variables provide local caching and are important for efficiency. They do however cause some issues when dealing with multiple subinterpreters, where the static local variable might have been initialized by one of the subinterpreters, and is not reset when accessed by another subinterpreter. More globally, they cannot be reset when the Numpy module is released, and thus will likely cause an issue if it is reloaded after being released. I have seen the issue mentionned in at least one pull request: https://github.com/numpy/numpy/pull/15169 and in several issues. If I understand correctly, the issue is not considered as important because subinterpreters are not yet prominent in CPython, static local variables provide an important service in caching data locally (instead of exposing these variables globally). So the benefits outweigh the costs and risks (that would be a huge change to the code base). I happen to maintain, compile and run a version of Python on iOS (https://github.com/holzschu/a-shell/ or https://apps.apple.com/us/app/a-shell/id1473805438), where I have to remove all these static local variables, because of the specificity of the platform (in order to run Python multiple times, I have to release and reset all modules). Right now, I'm maintaining the changes to the code base in a separate branch (https://github.com/holzschu/numpy/) and not necessarily in a very clean way. With the recent renewed interest in subinterpreters, I was wondering if there was a way I could contribute these changes back to the main numpy branch. I would have to clean up the code, obviously, and probably get guidance on how to do it cleanly, but the first question is: would there be an interest, or is that something I should keep in my separate branch? From a technical point of view, about 80% of these static local variables are just before a call to npy_cache_import(), and the most efficient way to do it (in terms of lines of code) is just to remove the part where npy_cache_import uses the static local variable. You pay a price in performance, but gain in usability. Best regards, Nicolas Holzschuch

3 2

Find location of slice in it's base
by Dom Grigonis Aug. 31, 2023

Aug. 31, 2023

Hi everyone, I am working with shared arrays and their slices. And trying to subclass ndarray in such way so that they remap to memory on unpickling. I am using package SharedArray, which doesn’t micro-manage memory locations, but rather stores the whole map via shm using unique name. One issue I ran into is that when I pickle & unpickle slice of memory mapped array, I need to store the spec of the subset so that I can retrieve the same portion. The issue I am working on has been briefly discussed on stack: https://stackoverflow.com/questions/12421770/find-location-of-slice-in-nump… <https://stackoverflow.com/questions/12421770/find-location-of-slice-in-nump…>. Although I am not sure if it is the exact same one. So, in short, is there a way to determine location of a slice in original array. Ideally, it would be indexing which works for any non-copy slice/subset possible. Kind regards, Dg —Nothing ever dies, just enters the state of deferred evaluation—

3 4

reminder: put an upper bound on setuptools if you use numpy.distutils!
by Ralf Gommers Aug. 31, 2023

Aug. 31, 2023

Hi all, After setuptools 65.0 was released a few days ago, all users of numpy.distutils had their builds broken. This is already fixed in setuptools 65.0.2 because the breakage was particularly bad. However, the next breakage may not be fixed anymore (and more breakages *are* expected). So this is a good time to remind you all that you should put an upper bound on the setuptools version you allow in the releases of your package - to the last version that is known to work with your package. Our official stance here is that setuptools versions >=60 are not supported - see the "deprecating numpy.distutils" thread: https://mail.python.org/archives/list/numpy-discussion@python.org/message/P…. Newer versions may work for you, depending on what features you use. They don't for NumPy and for SciPy; both projects pin to 59.X to avoid problems. For the recent issue with setuptools 65.0.0, see https://github.com/numpy/numpy/issues/22135. We have also made the warnings about this topic in our docs more explicit, see https://github.com/numpy/numpy/pull/22154. Cheers, Ralf

2 2

Suggestions for changes to the polynomial module
by jsdodge＠sfu.ca Aug. 29, 2023

Aug. 29, 2023

Hello, Since this is my first post to this group, I want to start by expressing my appreciation for the work that everyone has done to develop and maintain NumPy. Your work enabled me to adopt a fully open-source workflow in my research, after being a MATLAB user since v4. The quality of the NumPy/SciPy/Matplotlib ecosystem also helped me convince my department to standardize our courses around these libraries. Thank you! Now for my gripes.... ;) I've encountered two issues with the numpy.polynomial module that I'd like to synthesize here, since they are scattered among multiple GitHub issues that span over five years. I'm interested in this module because I've prepared some educational resources for using Python to do data analysis in the physical sciences (https://github.com/jsdodge/data-analysis-python), and these include how to use numpy.polyfit for fitting polynomial models to data (specifically, in https://github.com/jsdodge/data-analysis-python/blob/main/notebooks/5.1-Lin…). I had hoped to adopt either numpy.polynomial.polynomial.polyfit or numpy.polynomial.polynomial.Polynomial.fit by now, but currently neither of these functions serves as a suitable substitute. My concerns are more urgent now that numpy.polyfit has been deprecated. 1. The numpy.polyfit function returns the covariance matrix for the fit parameters, which I think is an essential feature for a function like this (see https://github.com/numpy/numpy/pull/11197#issuecomment-426728143). Currently, neither numpy.polynomial.polynomial.polyfit nor numpy.polynomial.polynomial.Polynomial.fit provide this option. 2. Both numpy.polyfit function and numpy.polynomial.polynomial.polyfit return fit parameters that correspond to the standard polynomial representation (ie, a_0 x^p + a_1 x^{p-1} + ... a_p for numpy.polyfit and x^p a_0 + a_1 x + ... a_p x^p for numpy.polynomial.polynomial.polyfit). The recommended method, Polynomial.fit, returns coefficients for a rescaled domain, and expects the user to distinguish between the standard representation and the rescaled form. Among other things, this decision leads to problems like the discrepancy between [7] and [8] below: In [3]: x = np.linspace(0, 100) In [4]: y = 2 * x + 17 In [5]: p = np.polynomial.Polynomial.fit(x, y, 1) In [6]: p Out[6]: Polynomial([117., 100.], domain=[ 0., 100.], window=[-1., 1.], symbol='x') In [7]: print(p) 117.0 + 100.0·x In [8]: print(p.convert()) 17.0 + 2.0·x The output of [7] is bad and the output of [8] is good. In my view, the ideal solution to (1) and (2) would be the following: a. Revise both numpy.polynomial.polynomial.polyfit and numpy.polynomial.polynomial.Polynomial.fit to compute the covariance matrix, either by default or as part of the output when full==True. The covariance matrix should be in the basis of the standard polynomial coefficients, not the rescaled ones. b. Revise numpy.polynomial.polynomial.Polynomial.fit to yield coefficients for the unscaled domain (as in [8] above) by default, while retaining the scaled coefficients internally for evaluating the fitted polynomial, transformation the fitted polynomial to another polynomial basis, etc. The covariance matrix should also be given in terms of the standard polynomial representation, not the representation for the rescaled domain. Thanks for your attention, Steve As an example, here is some code that I would like to refactor using the numpy.polynomial package. # Fit the data using known parameter uncertainties p, V = np.polyfit(current, voltage, 1, w=1 / alpha_voltage, cov="unscaled") # Print results print(f"Intercept estimate: ({1000*p[1]:.0f} ± {1000*np.sqrt(V[1][1]):.0f}) µV") print(f"Slope estimate: ({1000*p[0]:.2f} ± {1000*np.sqrt(V[0][0]):.2f}) Ω")

2 3