Mailman 3 November 2022 - NumPy-Discussion

deprecating the function np.round_
by Inessa Pawson 19 Nov '22

19 Nov '22

In NumPy, there are several functions that allow even rounding to a given number of decimals: np.around, np.round, np.round_. The function np.round_ is a disrecommended backwards-compatibility alias of the first two. I propose deprecating np.round_ to avoid further user confusion. To provide feedback on the proposed, please leave your comments in the issue on GitHub: https://github.com/numpy/numpy/issues/22617. -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

1 0

NEP 50: deprecating np.find_common_type
by Inessa Pawson 17 Nov '22

17 Nov '22

As part of the work outlined in NEP 50, the function np.find_common_type will be deprecated soon after the release of NumPy 1.24. To provide feedback on this decision, please respond in the Conversation section of the pull request: https://github.com/numpy/numpy/pull/22539. -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

1 0

New methods exposing UFunc dtype resolution/promotion and internals
by Sebastian Berg 16 Nov '22

16 Nov '22

Hi all, I would like to expose more of the ufunc internals in the following PR: https://github.com/numpy/numpy/pull/22422/ There are three new proposed functions. I hope the first one can be generally useful while the last two are very specific (and thus underscored), but will hopefully become useful e.g. for Numba or numexpr. ufunc.resolve_dtypes(dtypes, *, signature=None, casting=None, reduction=False) ----------------------------------------------------------- Allows you to find out what dtypes NumPy's implementation will use without executing a ufunc. For the full docs, see: https://output.circle-artifacts.com/output/job/c8f72dd5-f8fb-448c-8fd8-d618… Example from the docs: >>> int32 = np.dtype("int32") >>> float32 = np.dtype("float32") The typical ufunc call does not pass an output dtype. `np.add` has two inputs and one output, so leave the output as ``None`` (not provided): >>> np.add.resolve_dtypes((int32, float32, None)) (dtype('float64'), dtype('float64'), dtype('float64')) The loop found uses "float64" for all operands (including the output), the first input would be cast. ``resolve_dtypes`` supports "weak" handling for Python scalars by passing ``int``, ``float``, or ``complex``: >>> np.add.resolve_dtypes((float32, float, None)) (dtype('float32'), dtype('float32'), dtype('float32')) Where the Python ``float`` behaves samilar to a Python value ``0.0`` in a ufunc call. (See :ref:`NEP 50 <NEP50>` for details.) ufunc._resolve_dtypes_and_context(dtypes, *, signature=None, casting=None, reduction=False) ------------------------------------------------------------- Identical to the above, but it additionally returns a "call_info" which allows access to the actual ufunc implmentation. ufunc._get_loop(call_info, /, *, fixed_strides=None) ---------------------------------------------------- Second function that is passed the `call_info` from the previous one. Both would normally be called (this is because it is the way NumPy must do it internally and allows most flexibility). After doing both calls, `call_info` can be used from C to directly access the C implementation. Flux in the C-API are expected (for now). But for example Numba already releases new versions when NumPy releases a new version. Cheers, Sebastian

1 1

help wanted to review Portuguese translation
by Inessa Pawson 15 Nov '22

15 Nov '22

Angélica Cardozo took initiative to translate into Portuguese the subtitles for the video “Find your way in the NumPy codebase” ( https://www.youtube.com/watch?v=mTWpBf1zewc) posted on the official NumPy YouTube channel (https://www.youtube.com/c/NumPy_team). We are currently looking for a contributor who could review the translation. If you are available, please respond to this message in a thread. -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

1 0

A Strange Phenomenon with respect to Multi-threading
by 腾刘 14 Nov '22

14 Nov '22

Hello~, I'm here again. Recently I came up with a really weird and confusing phenomenon. I used to dig deep into Numpy's C code to find out how np.add(a, b) is implemented and I thought I've figured it out. As far as I know, there is no multi-threading in Numpy for addition, instead, we use SIMD to speed it up. However, I found that for some certain situation with certain version of Numpy, there is multi-threading. Here is the screenshot (where we can see multi-threading clearly): [image: image.png] The environment is: Python 3.6.5 :: Anaconda, Inc. Numpy 1.14.3 np.show_config(): mkl_info: libraries = ['mkl_rt', 'pthread'] library_dirs = ['/home/SIXIE/tliu/anaconda3/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/SIXIE/tliu/anaconda3/include'] blas_mkl_info: libraries = ['mkl_rt', 'pthread'] library_dirs = ['/home/SIXIE/tliu/anaconda3/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/SIXIE/tliu/anaconda3/include'] blas_opt_info: libraries = ['mkl_rt', 'pthread'] library_dirs = ['/home/SIXIE/tliu/anaconda3/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/SIXIE/tliu/anaconda3/include'] lapack_mkl_info: libraries = ['mkl_rt', 'pthread'] library_dirs = ['/home/SIXIE/tliu/anaconda3/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/SIXIE/tliu/anaconda3/include'] lapack_opt_info: libraries = ['mkl_rt', 'pthread'] library_dirs = ['/home/SIXIE/tliu/anaconda3/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/SIXIE/tliu/anaconda3/include'] However, when I try to repeat the multi-threading with my built-from-source Numpy, it failed (the cpu percentage is at most 100%): [image: image.png] And from the screenshot we can see that we also have mkl_rt and pthread when building the Numpy since I added it to *site.cfg*. The complete config info is: blas_armpl_info: NOT AVAILABLE blas_mkl_info: libraries = ['mkl_rt', 'pthread', 'mkl_rt'] library_dirs = ['/home/tliu/anaconda3/envs/numpy-dev/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/tliu/anaconda3/envs/numpy-dev/include'] blas_opt_info: libraries = ['mkl_rt', 'pthread', 'mkl_rt'] library_dirs = ['/home/tliu/anaconda3/envs/numpy-dev/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/tliu/anaconda3/envs/numpy-dev/include'] lapack_armpl_info: NOT AVAILABLE lapack_mkl_info: libraries = ['mkl_rt', 'pthread', 'mkl_rt'] library_dirs = ['/home/tliu/anaconda3/envs/numpy-dev/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/tliu/anaconda3/envs/numpy-dev/include'] lapack_opt_info: libraries = ['mkl_rt', 'pthread', 'mkl_rt'] library_dirs = ['/home/tliu/anaconda3/envs/numpy-dev/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/tliu/anaconda3/envs/numpy-dev/include'] Supported SIMD extensions in this NumPy install: baseline = SSE,SSE2,SSE3 found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2 not found = AVX512F,AVX512CD,AVX512_KNL,AVX512_KNM,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL I want to repeat multi-threading because I really want to know in which part of C code Numpy dispatches this computation to an outer library which utilize multi-threading (Numpy itself doesn't have multi-threading, if we see multi-threads, it must be created by outer library like BLAS, mkl, etc. But we know BLAS doesn't have functions for adding two vectors, so there must be mkl. BLAS is for matrix multiplication). But with pip-installed Numpy*, *I can't debug with it; I could only use *gdb *with my own built-from-source Numpy. I must miss something so that multi-threading doesn't recur. Any instructions or suggestions? Thank to all you guys in advance~

2 2

plan for moving to Meson
by Ralf Gommers 11 Nov '22

11 Nov '22

Hi all, With distutils now removed from the stdlib in the Python 3.12 release cycle, the clock is ticking a bit for dealing with our build system situation. With SciPy's move to Meson now basically complete - there are always loose ends & improvements, but the 1.9 releases have gone well - it's time to look at NumPy doing the same thing. Scikit-image has also merged support for Meson, and Pandas is about to. So all the generic & Python packaging related things we need should be in place. NumPy does have some hairy stuff of course that those other projects don't have (lots of config checks and platform-specific behavior, extensive SIMD support). It shouldn't be too difficult to get a baseline build - Linux/macOS with baseline SIMD flags - working, but there'll be a long tail of things that are hard to test or will need to be upstreamed to Meson. Here is a tracking issue where I wrote up a plan for how to approach the transition: https://github.com/numpy/numpy/issues/22546. Linked from there is a GitHub project board that we plan to use to divide up the work. And there is a `meson` branch in the repo with a start to the implementation. NumPy should default to building with Meson in the 1.25.0 release, which should also still build with `numpy.distutils`. And `numpy.distutils` will continue to be shipped for Python <3.12 for a while after that, until we determine that it's no longer needed. If anyone has ideas or concerns regarding the current plan, it'd be great to hear them - here or on the issue. Cheers, Ralf

4 7

Re: New feature: binary (arbitrary base) rounding
by Stefano Miccoli 11 Nov '22

11 Nov '22

On 8 Nov 2022, at 15:32, numpy-discussion-request(a)python.org<mailto:numpy-discussion-request@python.org> wrote: Thanks for the proposal. I don't have much of an opinion on this and right now I am mainly wondering whether there is prior art which can inform us that this is relatively widely useful? Base-2 (bit) rounding is implemented in Numcodecs <https://numcodecs.readthedocs.io/en/stable/bitround.html#module-numcodecs.b…> in the context of data compression. As pointed out by M. Klöwer et al. in <https://doi.org/10.1038/s43588-021-00156-2> "Rounding bits without real information to zero facilitates lossless compression algorithms and encodes the uncertainty within the data itself." I'm not an expert, but I never encountered rounding floating point numbers in bases different from 2 and 10. Stefano

5 9

New feature: binary (arbitrary base) rounding
by Oscar Gustafsson 10 Nov '22

10 Nov '22

Hi all, I hope this is the correct way to propose a new feature. https://github.com/numpy/numpy/issues/22522 Currently, the around-function supports rounding to a given number of decimal digits. It is often quite convenient to be able to round to a given number of binary digits to mimic fixed-point representations. Currently this can be readily achieved using e.g. fractional_bits = 5 scale = 2**fractional_bits x_round = np.around(x*scale)/scale However, it would be more convenient (and probably faster) to provide dedicated support for it. I see a few different ways to obtain this: 1. Provide a separate function (binaryround?) 2. Provide a base argument to around which defaults to 10. 3. Provide a quant(ization) function where the argument is the step-size. (For completeness, one may think of having multiple quantization modes, not just rounding) Any opinions? BR Oscar Gustafsson

3 3

NEP 50 and integers (e.g. uint8 together -1)
by Sebastian Berg 08 Nov '22

08 Nov '22

Hi all, In NEP 50 (https://numpy.org/neps/nep-0050-scalar-promotion.html) my current proposal is that the following: np.array([1, 2], dtype="uint8") + (-1) which currently returns an "int16" array must raise an error. This is because we want to return a uint8 array but the -1 cannot be represented well by -1. The same should also happen for a value of 300 (rather than -1). My main question is not about making this an error though. The question is whether the following two cases should also error: uint8_arr = np.array([1, 2], dtype="uint8") uint8_arr[0] = -1 or: np.array([-1], dtype=np.uint8) (In practice these currently give the maximum integer with well defined integer overflow.) Note that the call `np.uint8(-1)` could be a special case here! The reason I ask is that my PR: https://github.com/numpy/numpy/pull/21875 currently introduces the new error without changing the other cases and this requires adding a full new conversion path. If we want to change that anyway (or keep things aligned), I could simplify the logic in the PR. Cheers, Sebastian

3 5

New API for testing custom array containers
by Nathan 04 Nov '22

04 Nov '22

Hi all, I just opened a PR (https://github.com/numpy/numpy/pull/22533) that makes a minor modification to the numpy API. The PR creates numpy.testing.overrides, which contains some helpers for downstream projects who want to test types that implement __array_function__ and __array_ufunc__. See the discussion in https://github.com/numpy/numpy/issues/15544 for more context. -Nathan

1 0

NumPy-Discussion November 2022