Mailman 3 August 2021 - NumPy-Discussion

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem 18 Feb '23

18 Feb '23

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Documentation Team meeting - Monday June 8th
by Melissa Mendonça 04 Dec '22

04 Dec '22

Hi all! A reminder that on Monday, June 8, we have another documentation team meeting at 3PM UTC**. If you wish to join on Zoom, you need to use this link https://zoom.us/j/420005230 Here's the permanent hackmd document with the meeting notes: https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg <https://www.google.com/url?q=https%3A%2F%2Fhackmd.io%2FoB_boakvRqKR-_2jRV-Q…> Hope to see you around (especially if you want to introduce yourself or discuss ideas for Google Season of Docs). ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Documentati… - Melissa

6 74

copy="never" discussion and no deprecation cycle?
by Sebastian Berg 22 Jul '22

22 Jul '22

Hi all, (sorry for the length, details/discussion below) On the triage call, there seemed a preference to just try to skip the deprecation and introduce `copy="never"`, `copy="if_needed"`, and `copy="always"` (i.e. string options for the `copy` keyword argument). Strictly speaking, this is against the typical policy (one year of warning/errors). But nobody could think of a reasonable chance that anyone actually uses it. (For me just "policy" will be enough of an argument to just take it slow.) BUT: If nobody has *any* concerns at all, I think we may just end up introducing the change right away. The PR is: https://github.com/numpy/numpy/pull/19173 ## The Feature There is the idea to add `copy=never` (or similar). This would modify the existing `copy` argument to make it a 3-way decision: * `copy=always` or `copy=True` to force a copy * `copy=if_needed` or `copy=False` to prefer no-copy behavior * `copy=never` to error when no-copy behavior is not possible (this ensures that a view is returned) this would affect the functions: * np.array(object, copy=...) * arr.astype(new_dtype, copy=...) * np.reshape(arr, new_shape, copy=...), and the method arr.reshape() * np.meshgrid and possibly Where `reshape` currently does not have the option and would benefit by allowing for `arr.reshape(-1, copy=never)`, which would guarantee a view. ## The Options We have three options that are currently being discussed: 1. We introduce a new `np.CopyMode` or `np.<something>.Copy` Enum with values `np.CopyMode.NEVER`, `np.CopyMode.IF_NEEDED`, and `np.CopyMode.ALWAYS` * Plus: No compatibility concerns * Downside(?): This would be a first in NumPy, and is untypical API due to that. 2. We introduce `copy="never"`, `copy="if_needed"` and `copy="always"` as strings (all other strings will be a `TypeError`): * Problem: `copy="never"` currently means `copy=True` (the opposite) Which means new code has to take care when it may run on older NumPy versions. And in theory could make old code return the wrong thing. * Plus: Strings are the typical for options in NumPy currently. 3. Same as 2. But we take it very slow: Make strings an error right now and only introduce the new options after two releases as per typical deprecation policy. ## Discussion We discussed it briefly today in the triage call and we were leaning towards strings. I was honestly expecting to converge to option 3 to avoid compatibility issues (mainly surprises with `copy="never"` on older versions). But considering how weird it is to currently pass `copy="never"`, the question was whether we should not change it with a release note. The probability of someone currently passing exactly one of those three (and no other) strings seems exceedingly small. Personally, I don't have a much of an opinion. But if *nobody* voices any concern about just changing the meaning of the string inputs, I think the current default may be to just do it. Cheers, Sebastian

12 28

[Feature Request] Add alias of np.concatenate as np.concat
by Iordanis Fostiropoulos 11 May '22

11 May '22

In regard to Feature Request: https://github.com/numpy/numpy/issues/16469 It was suggested to sent to the mailing list. I think I can make a strong point as to why the support for this naming convention would make sense. Such as it would follow other frameworks that often work alongside numpy such as tensorflow. For backward compatibility, it can simply be an alias to np.concatenate I often convert portions of code from tf to np, it is as simple as changing the base module from tf to np. e.g. np.expand_dims -> tf.expand_dims. This is done either in debugging (e.g. converting tf to np without eager execution to debug portion of the code), or during prototyping, e.g. develop in numpy and convert in tf. I find myself more than at one occasion to getting syntax errors because of this particular function np.concatenate. It is unnecessarily long. I imagine there are more people that also run into the same problems. Pandas uses concat (torch on the other extreme uses simply cat, which I don't think is as descriptive).

7 6

request to remove the numpy-aarch64 package from PyPI
by Ralf Gommers 14 Mar '22

14 Mar '22

Hi all, FYI, I noticed this package that claimed to be maintained by us: https://pypi.org/project/numpy-aarch64/. That's not ours, so I tried to contact the author (no email provided, but guessed the same username on GitHub) and asked to remove it: https://github.com/tomasriv/DNA_Sequence/issues/1. There are a very large number of packages with "numpy" in the name on PyPI, and there's no way we can audit/police that effectively, but if it's a rebuild that pretends like it's official then I think it's worth doing something about. It could contain malicious code for all we know. Cheers, Ralf

5 7

Fwd: ndarray should offer __format__ that can adjust precision
by Ivan Gonzalez 04 Dec '21

04 Dec '21

It would be nice to be able to use the Python syntax we already use to format the precision of floating numbers in numpy: >>> a = np.array([-np.pi, np.pi]) >>> print(f"{a:+.2f}") [-3.14 +3.14] This is particularly useful when you have large arrangements. The problem is that if you want to do it today, it is not implemented: >>> print(f"{a:+.2f}") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported format string passed to numpy.ndarray.__format__ In this PR (https://github.com/numpy/numpy/pull/19550) I propose a very basic formatting implementation for numeric numbers that uses `array2string` just like it currently does `str` At first, since we are only considering formatting the numeric type, floating numbers specifically, we are only interested in being able to change the precision, the sign, and possibly the rounding or truncation. Since the `array2string` function already does everything we need, we only need to implement the` __format__` function of the `ndarray` class which parses a predefined format (similar to the one already used by Python for built-in data types) to indicate the parameters before said. I propose a mini format specification inspired in the [Format Specification Mini-Language](https://docs.python.org/3/library/string.html#formatspec). ``` format_spec ::= [sign][.precision][type] sign ::= "+" | "-" | " " precision ::= [0-9]+ type ::= "f" | "e" ``` We are going to consider only 3 arguments of the `array2string` function:` precision`, `suppress_small`,` sign`. In particular, the `type` token sets the` suppress_small` argument to True when the type is `f` and False when it is `e`. This is in order to mimic Python's behavior in truncating decimals when using the fixed-point notation. As @brandon-rhodes said in gh-5543, the behavior when you try to format an array containing Python objects, the behavior should be the same as Python has implemented by default in the `object` class: ` format (a, "") ` should be equivalent to `str (a)` and `format(a, "not empty")` should raise an exception. What remains to be defined is the behavior when trying to format an array with a non-numeric data type (`np.numeric`) other than `np.object_`. Should we raise an exception? In my opinion yes, since in the future formatting is extended -- for example, for dates -- people are aware that before that was not implemented. I'm open to suggestions. - Ivan

3 2

Newcomer's meeting - later today (4pm UTC)!
by Melissa Mendonça 01 Dec '21

01 Dec '21

Hi all! Sorry for the late notice - our next Newcomer's Meeting is today, * July 15, at 4pm UTC.* This is an informal meeting with no agenda to ask questions, get to know other people and (hopefully) figure out ways to contribute to NumPy. Feel free to join if you are lurking around but found it hard to start contributing - we'll do our best to support you. If you wish to join on Zoom, use this link: https://zoom.us/j/6345425936 Hope to see you around! ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Newcomer%27… <https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Newcomer%27…> *** You can add the NumPy community calendar to your google calendar by clicking this link: https://calendar.google.com/calendar /r?cid=YmVya2VsZXkuZWR1X2lla2dwaWdtMjMyamJobGRzZmIyYzJqODFjQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20 - Melissa

1 7

Revert the return of a single NaN for `np.unique` with floating point numbers?
by Sebastian Berg 14 Nov '21

14 Nov '21

Hi all, In NumPy 1.21, the output of `np.unique` changed in the presence of multiple NaNs. Previously, all NaNs were returned when we now only return one (all NaNs were considered unique): a = np.array([1, 1, np.nan, np.nan, np.nan]) Before 1.21: >>> np.unique(a) array([ 1., nan, nan, nan]) After 1.21: array([ 1., nan]) This change was requested in an old issue: https://github.com/numpy/numpy/issues/2111 And happened here: https://github.com/numpy/numpy/pull/18070 While, it has a release note. I am not sure the change got the attention it deserved. This would be especially worrying if it is a regression for anyone? Cheers, Sebastian PS: One additional note, is that this does not work for object arrays (it cannot reasonable): >>> np.unique(a.astype(object)) array([1.0, nan, nan, nan], dtype=object)

4 6

A bite of C++
by Serge Guelton 08 Oct '21

08 Oct '21

Hi folks, https://github.com/numpy/numpy/pull/19713 showcases what *could* be a first step toward getting rid of generated C code within numpy, in favor of some C++ code, coupled with a single macro trick. Basically, templated code is an easy and robust way to replace generated code (the C++ compiler becomes the code generator when instantiating code), and a single X-macro takes care of the glue with the C world. Some changes in distutils were needed to cope with C++-specific flags, and extensions that consist in mixed C and C++ code. I've kept the change as minimal as possible to ease the (potential) transition and keep the C++ code close to the C code. This led to less idiomatic C++ code, but I value a "correct first" approach. There's an on-going effort by seiko2plus to remove that C layer, I acknowledge this would bring even more C++ code, but that looks orthogonal to me (and a very good second step!) All lights are green for the PR, let's assume it's a solid ground for discussion :-) So, well, what do you think? Should we go forward? Potential follow-ups : - do we want to use -nostdlib, to be sure we don't bring any C++ runtime dep? - what about -fno-exception, -fno-rtti? - coding style? - (I'm-not-a-farseer-I-don-t-know-all-topics)

6 9

New CZI grant to support DEI initiatives in the scientific Python ecosystem
by Melissa Mendonça 02 Sep '21

02 Sep '21

We are happy to announce the Chan Zuckerberg Initiative has awarded a grant to support the onboarding, inclusion, and retention of people from historically marginalized groups on scientific Python projects, and to structurally improve the community dynamics for NumPy, SciPy, Matplotlib, and Pandas. As a part of CZI’s Essential Open Source Software for Science program [1], this Diversity & Inclusion supplemental grant [2] will support the creation of dedicated Contributor Experience Lead positions to identify, document, and implement practices to foster inclusive open-source communities. This project will be led by Melissa Mendonça (NumPy), with additional mentorship and guidance provided by Ralf Gommers (NumPy, SciPy), Hannah Aizenman and Thomas Caswell (Matplotlib), Matt Haberland (SciPy), and Joris Van den Bossche (Pandas). This is an ambitious project aiming to discover and implement activities that should structurally improve the community dynamics of our projects. By establishing these new cross-project roles, we hope to introduce a new collaboration model to the Scientific Python communities, allowing community-building work within the ecosystem to be done more efficiently and with greater outcomes. We also expect to develop a clearer picture of what works and what doesn’t in our projects to engage and retain new contributors, especially from historically underrepresented groups. Finally, we plan on producing detailed reports on the actions executed, explaining how they have impacted our projects in terms of representation and interaction with our communities. The two-year project is expected to start by November 2021, and we are excited to see the results from this work! You can read the full proposal in figshare [3] and see this announcement at the NumPy site [4]. Cheers! - Melissa [1] https://chanzuckerberg.com/eoss/ [2] https://cziscience.medium.com/advancing-diversity-and-inclusion-in-scientif… [3] https://figshare.com/articles/online_resource/Advancing_an_inclusive_cultur… [4] https://numpy.org/news/#advancing-an-inclusive-culture-in-the-scientific-py…

2 1

NumPy-Discussion August 2021