Mailman 3 February 2022 - NumPy-Discussion

NumPy community meeting – Monday, February 28th, 2022 at 16:00 (4 pm) UTC
by Inessa Pawson 27 Feb '22

27 Feb '22

The next NumPy documentation meeting is scheduled for this Monday, February 28th at 16:00 (4 pm) UTC. Join us via Zoom: https://zoom.us/j/96219574921?pwd=VTRNeGwwOUlrYVNYSENpVVBRRjlkZz09 Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg -- Cheers, Inessa *Inessa Pawson* Contributor Experience Lead | NumPy email: inessa(a)albuscode.org

1 0

NumPy Newcomers' Hour – Thursday, February 24th
by Inessa Pawson 23 Feb '22

23 Feb '22

Our next Newcomers' Hour is tomorrow, February 24th at 4 pm UTC. We have no agenda this time. Stop by to ask questions or just to say hi. Join the meeting via Zoom: https://us02web.zoom.us/j/87192457898 Cheers, Inessa Inessa Pawson NumPy Contributor Experience Lead

1 0

Feature query: fetch top/bottom k from array
by Joseph Bolton 23 Feb '22

23 Feb '22

Morning, My apologies if this deviates from the vision of numpy: I find myself often requiring the indices and/or values of the top (or bottom) k items in a numpy array. I am aware of solutions involving partition/argpartition but these are inelegant. I am thinking of 1-dimensional arrays, but this concept extends to an arbitrary number of dimensions. Is this a feature that would benefit the numpy package? I am happy to code it up. Thanks for your time! Best regards Joe

4 4

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg 22 Feb '22

22 Feb '22

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, February 23rd at 17:00 UTC (9:00am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. Best regards Sebastian

1 0

Re: NEP draft for the future behaviour of scalar promotion
by Juan Nunez-Iglesias 22 Feb '22

22 Feb '22

On Tue, 22 Feb 2022, at 1:01 AM, Stefan van der Walt wrote: > it is easier to explain away `x + 1` behaving oddly over `x[0] + 1` behaving oddly Is it? I find the two equivalent, honestly. > given that we pretend like NumPy scalars do not exist. This is the leaky abstraction that I think should be plugged in this revamp. > This then argues for making explicit to the user that there are scalars involved. I.e., no more: > > In [4]: x = np.array([1, 2, 3]) > > In [5]: x[0] > Out[5]: 1 > > But rather > > Out[5]: np.int64(1) Yup. I would be in favour of such a repr change. (And to be clear, it is *only* a repr change, not a behaviour change!) I have indeed run across this a few times, e.g. trying to encode a single value in json only to find that it was a NumPy int64 rather than an int. >>> The benefit of these semantics are that you can readily express sequences of operations with clean Python code, without having to explicitly cast scalars to the appropriate type. Imagine if rather than writing this: >>> >>> 3 * (x + 1) ** 2 >>> you had to write this: >>> >>> np.int32(3) * (x + np.int32(1)) ** np.int32(2) > > And how do you write the much more common > > x[0] + 1 Is it really much more common than arithmetic combining arrays and literals? I'd say it's much *less* common, especially in "idiomatic" NumPy which tries to avoid Python looping over elements. > now? It becomes: x[0] + np.int64(1). I would write it as x[0].astype(np.int64) + 1, and indeed I think I would find that less confusing, reading the code years later, because it would allow me to not even have to think about type promotion. > The reason we had value inspection was that it gave us a cushy "best of both worlds"; when going with dtype-only casting, you have to give something up. Yes yes, we agree we are giving something up, we merely disagree about what is better to give up long term for our community. For me, the attractiveness of unified scalar and array semantics, together with unified type promotion, beats the attractiveness of hiding overflow from users, especially since the hiding can only ever be patchy.* I 100% agree with you that it is a tradeoff. But, imho, one worth making. * e.g. the same user might initially be happy about the result of x[0] + 1 matching their infinite-precision expectation, but then be surprised by x[0] + 1 -> 256 y[0] = 1 x[0] + y[0] -> 0 # WTH Juan.

2 2

NEP draft for the future behaviour of scalar promotion
by Sebastian Berg 22 Feb '22

22 Feb '22

Hi all, NumPy has awkward behaviour when it comes to promotion with 0-D arrays, and Python scalars. This is both a technical challenge (numpy needs to inspect the values where it shouldn't), as well as surprising for users. Roughly speaking, I have made a proposal under the 3 points: * NumPy scalars and NumPy arrays always behave the same. * A NumPy array always respects the dtype * A Python scalar is "weak" so that uint8_arr + 3 returns a uint8_arr The NEP is here: https://25105-908607-gh.circle-artifacts.com/0/doc/neps/_build/html/nep-005… But please refer to the PR, since above may go away or get outdated: https://github.com/numpy/numpy/pull/21103 Note that I have not 100% made up my mind on these, because some alternatives exist which may give a somewhat easier transition. Because of this, this is a very early draft (expect large changes/rewrite), but some feedback/input may go a long way to make sure we keep moving on this project. For those aware of the issues, it probably makes sense to skip ahead to the "Alternatives" section. I do expect that a large refactor/rewrite will be necessary, but need some feedback to keep moving. I had send the poll recently: https://discuss.scientific-python.org/t/poll-future-numpy-behavior-when-mix… just to say, I have not completely ignored it, although (as expected) the results do not give a very simple answer. Many agree with the choices I made, but some also seem to prefer "strong" Python types, or more special handling of NumPy scalars. Please do not hesitate to give opinions! I am not sure we can find a clear "obviously right" solution. Especially since there are tough backwards compatibility choices (even if most users are likely not to notice). So any input is appreciated. Cheers, Sebastian

5 5

14ᵗʰ Advanced Scientific Programming in Python in Bilbao Spain, 5–11 September, 2022
by Tiziano Zito 21 Feb '22

21 Feb '22

ASPP2022: 14ᵗʰ Advanced Scientific Programming in Python ======================================================== a Summer School by the ASPP faculty and the Faculty of Engineering of the Mondragon University, Bilbao https://aspp.school Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists have been trained to use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques and best practices which are standard in the industry, but especially tailored to the needs of a programming scientist. Lectures are devised to be interactive and to give the students enough time to acquire direct hands-on experience with the materials. Students will work in pairs throughout the school and will team up to practice the newly learned skills in a real programming project — an entertaining computer game. We use the Python programming language for the entire course. Python works as a simple programming language for beginners, but more importantly, it also works great in scientific simulations and data analysis. We show how clean language design, ease of extensibility, and the great wealth of open source libraries for scientific computing and data visualization are driving Python to become a standard tool for the programming scientist. This school is targeted at Master or PhD students and Post-docs from all areas of science. Competence in Python or in another language such as Java, C/C++, MATLAB, or R is absolutely required. Basic knowledge of Python and of a version control system such as git, subversion, mercurial, or bazaar is assumed. Participants without any prior experience with Python and/or git should work through the proposed introductory material before the course. We are striving hard to get a pool of students which is international and gender-balanced. Date & Location =============== 5–11 September, 2022. Bilbao, Spain. Application =========== You can apply online: https://aspp.school Application deadline: 23:59 UTC, Sunday 1 May, 2022. There will be no deadline extension, so be sure to apply on time. Be sure to read the FAQ before applying: https://aspp.school/wiki/faq Participation is for free, i.e. no fee is charged! Participants however should take care of travel, living, and accommodation expenses by themselves. We are in the process of securing some funds for supporting students with accommodation and living costs. Program ======= • Version control with git and how to contribute to open source projects with GitHub • Best practices in data visualization • Testing and debugging scientific code • Advanced NumPy • Organizing, documenting, and distributing scientific code • Advanced scientific Python: context managers and generators • Writing parallel applications in Python • Profiling and speeding up scientific code with Cython and numba • Programming in teams Faculty ======= • Jakob Jordan, Department of Physiology, University of Bern Switzerland • Jenni Rinker, Department of Wind Energy, Technical University of Denmark, Roskilde Denmark • Lisa Schwetlick, Experimental and Biological Psychology, Universität Potsdam Germany • Nicolas Rougier, Inria Bordeaux Sud-Ouest, Institute of Neurodegenerative Diseases, University of Bordeaux France • Pamela Hathway, GfK, Nuremberg Germany • Pietro Berkes, NAGRA Kudelski, Lausanne Switzerland • Rike-Benjamin Schuppner, Institute for Theoretical Biology, Humboldt-Universität zu Berlin Germany • Tiziano Zito, Department of Psychology, Humboldt-Universität zu Berlin Germany • Zbigniew Jędrzejewski-Szmek, Red Hat Inc., Warsaw Poland Organizers ========== Head of the organization for ASPP and responsible for the scientific program: • Tiziano Zito, Department of Psychology, Humboldt-Universität zu Berlin Germany Organization team in Bilbao: • Aitor Morales-Gregorio, Theoretical Neuroanatomy, Institute of Neuroscience and Medicine (INM-6), Forschungszentrum Jülich, Germany • Carlos Cernuda, Data Analysis & Cybersecurity, Faculty of Engineering, Mondragon Unibertsitatea, Bilbao Spain Website: https://aspp.school Contact: info(a)aspp.school

1 0

advice for using datetime64 in C binding
by Benoit Gschwind 18 Feb '22

18 Feb '22

Hello, I using the following code: if (PyArray_TYPE(arr1) == NPY_DATETIME) { // Ensure datetime64[ms] auto tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_cast<PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); std::swap(arr1, tmp); Py_XDECREF(tmp); // Ensure integer tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_cast<PyObject*>(arr1), "astype", "(s)", "i8")); std::swap(arr1, tmp); Py_XDECREF(tmp); tmp = reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1, PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); std::swap(arr1, tmp); Py_XDECREF(tmp); } First, if something is wrong with my code let me known. Then I wonder if I can have a safe shortcut to avoid converting datetime64 to i8. I guess the internal data of datetime64[ms] is i8 thus copying and casting the array may be avoided. Thanks by advance Best regards.

2 8

adding a new member to the NumPy CoC committee
by Ralf Gommers 16 Feb '22

16 Feb '22

Hi all, Anirudh Subramanian has indicated that he is happy to be replaced on the Code of Conduct committee. Right now the committee is Stefan van der Walt, Melissa Mendonca, and Anirudh (stepping down). We'd like to add a new third person. If anyone is interested, could you please reply or reach out to Stefan/Melissa/me? For reference, the last time we updated the committee in May 2020: https://mail.python.org/archives/list/numpy-discussion@python.org/message/4… Cheers, Ralf

1 1

Unpacking 0-D object arrays when setting item/filling
by Sebastian Berg 15 Feb '22

15 Feb '22

Hi all, TL;DR: Are there strong opinions on what: arr = np.array([None]) arr[0] = np.array(3) arr.fill(np.array(3)) Should give? Right now NumPy usually stores the 0-D array, I want to make `arr.fill()` consistent, so that is what it would be doing also. **Long discussion:** NumPy has a bit of a consistency problem with item setting, that is, what should happen if you have an array and set an individual element: arr = np.arange(10, dtype=object) # object is most problematic arr[0] = value Or similarly (but currently somewhat different): arr.fill(value) I wish to make the behaviour more consistent here, that is, both of these should use the same logic (fill is slightly different currently) and that logic should be shared by all functionality that is based on "setting a single element". There is a tricky situation however if `value` is a 0-D array. In most cases, we can just copy the value over (using correct casting) as if we copied an array: arr[0] = zero_d_arr is the same as: arr[0, ...] = zero_d_arr # definitely copies the array value But this is currently *not* consistently the case. I wish to make this consistent. The confusion is around object arrays, though: value = np.array(None, dtype=object) arr[0] = value Stores `value` without unpacking it currently. arr.fill(value) Stores the `None` (unpacking `value`) if and only if `value` is 0-D. Right now, i aligned `fill` with the item assignment, but this could be argued. Further related behaviour is that: np.array(value) # unpacks any array np.array([value, None], dtype=object) # does not unpack Which allows you to pack arrays into object arrays. Now, we could unpack making it impossible to place a 0-D array into an object array except via `arr.itemset()`. My current preference is to store the 0-D arrays as-is, and basically say that passing a 0-D array as value for: arr.fill(0d_arr) arr[0] = 0d_arr # this is fine: arr[0, ...] = 0d_arr it makes some issues around "array-like" objects a bit easier, and is just what we do _most_ (but not all!) of the time. But I know that many will look at it and immediately say: No this has to be unpacked! So I am open to suggestions or changing this in general. (However, I think aligning `arr.fill` is OK, changing `arr[0] = ...`, I don't know.) Cheers, Sebastian

3 3