Mailman 3 January 2022 - NumPy-Discussion

numpy 1.22.1 failed with "RuntimeError: Setuptools version is '60.5.0', version < '60.0.0' is required."
by s.srigowthami＠gmail.com Jan. 18, 2022

Jan. 18, 2022

Hi, We are using Openstack zuul ( Openstack CI ) to deploy devstack on IBM PPC64el hardware In https://review.opendev.org/c/openstack/requirements/+/824810/2/upper-constr… commit the numpy package version is updated from 1.22.0 to 1.22.1 When the version 1.22.0 of numpy was used setuptools-60.5.0 has successfully deployed. where as with the latest numpy 1.22.1 the setuptools package installation is failing with the below runtime error. Error: 2022-01-18 12:17:18.663446 | devstack-focal-newcloud | Collecting numpy 2022-01-18 12:17:18.763054 | devstack-focal-newcloud | Downloading numpy-1.22.1.zip (11.4 MB) 2022-01-18 12:17:24.048280 | devstack-focal-newcloud | Installing build dependencies: started 2022-01-18 12:17:32.883181 | devstack-focal-newcloud | Installing build dependencies: finished with status 'done' 2022-01-18 12:17:32.890122 | devstack-focal-newcloud | Getting requirements to build wheel: started 2022-01-18 12:17:33.282029 | devstack-focal-newcloud | Getting requirements to build wheel: finished with status 'error' 2022-01-18 12:17:33.282475 | devstack-focal-newcloud | ERROR: Command errored out with exit status 1: 2022-01-18 12:17:33.282499 | devstack-focal-newcloud | command: /usr/bin/python3.8 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpkr3xl1i_ 2022-01-18 12:17:33.282529 | devstack-focal-newcloud | cwd: /tmp/pip-install-c1eqnkjm/numpy_7e0d04ecfb1b47af806fb5707c655744 ....... 2022-01-18 12:17:33.283155 | devstack-focal-newcloud | RuntimeError: Setuptools version is '60.5.0', version < '60.0.0' is required. See pyproject.toml ... 2022-01-18 12:17:49.575387 | devstack-focal-newcloud | 2022-01-18 12:17:49.575401 | devstack-focal-newcloud | The conflict is caused by: 2022-01-18 12:17:49.575422 | devstack-focal-newcloud | websockify 0.10.0 depends on numpy 2022-01-18 12:17:49.575450 | devstack-focal-newcloud | The user requested (constraint) numpy===1.22.1 numpy: 1.22.1 ubuntu: Focal python : 3.8 Could someone help me with a way forward. Thanks Gowthami

2 1

ndarray typing advise
by Vincent Schut Jan. 18, 2022

Jan. 18, 2022

Hi, I'm having a hard(ish) time adding the right type annotations to some numpy-using code. I hope someone more knowledgeable can give me some advise. Specifically, the following questions pop up: - How do I correctly type a ndarray as being an array of float64, such that mypy can infer that when I index this array, the result is actually a float (currently I consistently get "Any" after indexing)? - How can I get compatibility between np.floating and builtin.float? Currently, some numpy functions are typed to return np.floating[Any] (e.g. np.linalg.norm). When I feed this result into something that is supposed to receive a builtin.float, mypy complains. Is an explicit cast(float, <the np.floating[Any] thing>) the prefered way to gain compatibility here? versions: numpy: 1.22.0 python: 3.8.10 (unfortunately we're limited to this version) mypy: 0.931 Thanks! Vincent. -- Vincent Schut Remote Sensing Software Engineer +31 302272679 ~ Maliebaan 22 | 3581CP | Utrecht | Netherlands Linkedin <https://www.linkedin.com/company/satelligence/>~ satelligence.com <http://www.satelligence.com><http://www.satelligence.com>

3 3

Importance of setting up SVML during build
by Sandro Tosi Jan. 16, 2022

Jan. 16, 2022

Hello, i'm preparing 1.22.1 for inclusion in Debian, and noticed how the new numpy has a git submodule to include numpy/SVML in its source tree. We are basing the debian package on the github tarball, and that does not include any submodule, so i'm wondering how important it is to have the SVML files around when building the packages. Given it's not immediately possible to include an external set of files, i'm just trying to balance the effort to inject them and their usefulness. At the same time, i'm wondering if https://github.com/numpy/numpy/pull/20695 (the PR that introduced NPY_DISABLE_SVML, also probably https://github.com/numpy/numpy/pull/20741 for the doc) should be backported to the 1.22.x branch, so that we can start using it in the release that introduced that feature. Thanks, -- Sandro "morph" Tosi My website: http://sandrotosi.me/ Me at Debian: http://wiki.debian.org/SandroTosi Twitter: https://twitter.com/sandrotosi

2 2

MacPython travis funds have run out (again)
by Charles R Harris Jan. 14, 2022

Jan. 14, 2022

Hi All, Travis wheel build problems again. There is still a positive balance, but apparently it is insufficiently positive. Which is strange if ARM64 is free. Wheel builds have been failing since last weekend. Matti, could you ask for more credits? I'm thinking we really need to move off travis for ARM builds. Chuck

3 7

np.convolve question
by cae＠live.in Jan. 14, 2022

Jan. 14, 2022

Hello, I am a mechanical engineer and am using Numpy Convolve to smoothen 1D data collected from physical testing for a diesel engine. (I found convolve the best after comparing with other smoothing methodologies) I tried reading the numpy manual and the wikipedia article referenced in it. But I would specifically like to ask/discuss what would happen if the second 1D input is not given? what type of window will the function pass the data through to smoothen it? Here are the articles I have read to implement my smoothing code: References: 1. https://numpy.org/doc/stable/reference/generated/numpy.convolve.html 2. https://en.wikipedia.org/wiki/Convolution 3. https://stackoverflow.com/questions/20618804/how-to-smooth-a-curve-in-the-r… NOTE: Apologies if I have unknowingly broken any rule of this forum - I can delete the thread if it doesn't meet any set guidelines of the forum. Best, Pushkar Sheth Bangalore, India.

3 2

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg Jan. 12, 2022

Jan. 12, 2022

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, January 12th at 17:00 UTC (9:00am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. Best regards Sebastian

1 0

Writing an array subclass in a compiled language with proper dispatch
by Juan Luis Cano Rodríguez Jan. 11, 2022

Jan. 11, 2022

Hi all, I am a long time user of astropy.units, which allows one to define quantities with physical units as follows: >>> from astropy import units as u >>> 10 << u.cm <Quantity 10. cm> >>> np.sqrt(4 << u.m ** 2) <Quantity 2. m> >>> ([1, 1, 0] << u.m) @ ([0, 10, 20] << u.cm / u.s) <Quantity 10. cm m / s> >>> (([1, 1, 0] << u.m) * ([0, 10, 20] << u.cm / u.s)).to(u.m ** 2 / u.s) <Quantity [0. , 0.1, 0. ] m2 / s> The mechanism works by subclassing numpy.ndarray and leveraging __array_function__ support aka NEP 18. Internally it is something like this: >>> v = np.array(10, dtype=np.float64, copy=False, order=None, subok=True, ndmin=0) >>> vu = v.view(u.Quantity) >>> vu._set_unit(u.cm) >>> vu <Quantity 10. cm> However, over the years I have been constantly annoyed by the fact that it is tremendously slow. I'm not critizing Astropy devs, the problem seems objectively difficult: although some code paths could be optimized at the cost of losing some syntactic sugar or breaking backwards compatibility, `isinstance` calls and introspection in general are slow. Setting aside the question of trying to make astropy.units faster (which may or may not be possible), I was thinking how feasible could it be to implement something similar, but using a compiled language instead (C, Cython, Rust, whatever) and leveraging "modern" dispatch mechanisms. But after reading about NEP 18, NEP 47, uarray, and various pull requests and issues here and there (https://labs.quansight.org/blog/2021/11/pydata-extensibility-vision/ and https://github.com/scipy/scipy/issues/10204#issuecomment-787067947 among others) I don't fully grasp the differences between the approaches, and I don't know if what I am proposing is feasible at all. Since IIUC the numpy function or ufunc is passed to __array_function__ and __array_ufunc__ respectively, I am not sure how would that interact with the code being in a foreign language (I assume the NumPy C API would have to be used). If folks have advice, ideas, or suggestions for a direction, I'll be happy to read them. Best!

3 3

representation of valid float type range
by alejandro.giacometti＠gmail.com Jan. 10, 2022

Jan. 10, 2022

I am getting an interesting result, and I'm wondering if anyone would care to give me some intuition of why. The example is simple enough, I want to get a range of values that are representable by a type: ```python f64_info = np.finfo(np.float64) valid_range = np.linspace( start=f64_info.min, stop=f64_info.max, num=10 ) valid_range => array([ nan, inf, inf, inf, inf, inf, inf, inf, inf, 1.79769313e+308]) ``` The minimum value is representable by the type, I can see it: ```python f64_info.min => -1.7976931348623157e+308 ``` I thought that maybe the valid range cannot start with the minimun value, so I've tried a few alternatives: ```python valid_range = np.linspace( start=f64_info.min + f64_info.eps, stop=f64_info.max, num=10 ) valid_range => array([ nan, inf, inf, inf, inf, inf, inf, inf, inf, 1.79769313e+308]) valid_range = np.linspace( start=f64_info.min + f64_info.tiny, stop=f64_info.max, num=10 ) valid_range => array([ nan, inf, inf, inf, inf, inf, inf, inf, inf, 1.79769313e+308]) ``` I thought maybe the range is too wide, but I can do this: ```python valid_range = np.linspace( start=0, stop=f64_info.max, num=10 ) valid_range => array([0.00000000e+000, 1.99743682e+307, 3.99487363e+307, 5.99231045e+307, 7.98974727e+307, 9.98718408e+307, 1.19846209e+308, 1.39820577e+308, 1.59794945e+308, 1.79769313e+308]) ... valid_range = np.linspace( start=f64_info.tiny, stop=f64_info.max, num=10 ) valid_range => array([2.22507386e-308, 1.99743682e+307, 3.99487363e+307, 5.99231045e+307, 7.98974727e+307, 9.98718408e+307, 1.19846209e+308, 1.39820577e+308, 1.59794945e+308, 1.79769313e+308]) ... f32_info = np.finfo(np.float32) valid_range = np.linspace( start=f32_info.tiny, stop=f32_info.max, num=10, dtype=np.float32, ) valid_range => array([1.1754944e-38, 3.7809150e+37, 7.5618299e+37, 1.1342745e+38, 1.5123660e+38, 1.8904575e+38, 2.2685490e+38, 2.6466405e+38, 3.0247320e+38, 3.4028235e+38], dtype=float32) ``` I know that linear space is arbitrary, and perhaps not that useful. In fact this is valid: ```python valid_range = np.logspace( start=f64_info.minexp, stop=f64_info.maxexp, num=10, base=2, endpoint=False ) valid_range => array([2.22507386e-308, 8.67124674e-247, 3.37923704e-185, 1.31690901e-123, 5.13207368e-062, 2.00000000e+000, 7.79412037e+061, 3.03741562e+123, 1.18369915e+185, 4.61294681e+246]) ``` But I'm still confused on why linear space is invalid Thanks!

7 10

mypy false positives
by Vincent Schut Jan. 7, 2022

Jan. 7, 2022

Hi all, I've got a question about numpy typing. We have our own internal codebase that heavily depends on numpy, and we're trying to get the number of mypy errors down. Many of the remaining mypy errors are actually numpy false positives. I hoped that with the numpy 1.22.0 release these would be fixed, as there ought to be many typing improvements in that version, but apparently they're not? I'm asking here to check if this is just still a work in progress and we should be patient, or if there's something wrong on our side...? Some typical false positives are: "finfo[float32]" has no attribute "tiny" Module has no attribute "count_nonzero" Module has no attribute "rot90" Module has no attribute "asfortranarray" Module has no attribute "isfinite" Module has no attribute "any" Module has no attribute "histogram"; maybe "histogramdd"? Module has no attribute "bitwise_or" Module has no attribute "bitwise_and" Module has no attribute "indices" Module has no attribute "pad" versions: python 3.8.10; numpy 1.22.0; mypy 0.930 -- Vincent Schut Remote Sensing Software Engineer +31 302272679 ~ Maliebaan 22 | 3581CP | Utrecht | Netherlands Linkedin <https://www.linkedin.com/company/satelligence/>~ satelligence.com <http://www.satelligence.com><http://www.satelligence.com>

2 2

No credits left in travis for 1.22.0 release
by Charles R Harris Jan. 6, 2022

Jan. 6, 2022

Hi All, We need to get more credits in order to run MacPython/numpy-wheels builds on travis. Anyone recall how to go about this? Chuck

5 8