Mailman 3 October 2021 - NumPy-Discussion

Deprecate np.MachAr?
by bas van beek Nov. 8, 2021

Nov. 8, 2021

Hi all, The subject of `MachAr` recently came up in https://github.com/numpy/numpy/pull/20132 and an earlier community meeting, more notably: it’s questionably role as public (and even private?) API. Ever since the introduction of hard-coded `finfo` parameters back in numpy/numpy#8504 there has been progressively less need for computing machine parameters during runtime, to the point where `MachAr` is effectively unused in the numpy codebase itself[1]. From a user-API point of view, the main difference between ` finfo` and `MachAr` these days is that the latter produces the same results roughly 4 orders of magnitude slower than the former… Are there any thoughts about deprecating it? Regards, Bas [1] It’s still used as fallback for `longdouble` types with unknown bytes-representations, rare cases that probably deserve their own pull request.

3 2

Percentile/Quantile "interpolation" refactor
by Sebastian Berg Nov. 8, 2021

Nov. 8, 2021

Hi all, after a long time Abel has helped us and refactored the quantile and percentile functions' `interpolation` keyword. This was long overdue since NumPy implements three (the non-default) interpolation methods that appear to be very much non-standard. On the other hand, NumPy currently has no unbiased methods (i.e. population estimate). There are two main questions right now with respect to the API. First which names to use for the methods and second, how to deal with "outliers". The PR https://github.com/numpy/numpy/pull/19857#issuecomment-939852134 adds the methods and gives them (currently) the following names (sorted by the R methods) – the names will be used as string identifiers: 1. inverted cdf 2. averaged inverted cdf 3. closest observation 4. interpolated inverted cdf 5. hazen (name from wolfram) 6. weibull (name from wolfram) 7. linear (default! Better name deferred) 8. median unbiased 9. normal unbiased And additionally the four ones we currently have: * lower * higher * nearest * midpoint Number 5. and 6. are named "exclusive" and "inclusive" by Python in their `method` keyword argument. While I like the name `method=` and may want to move to it, I am not sure I like "inclusive" and "exclusive". The current plan was to defer the kwarg rename into a followup, although it should be discussed before the next release. The second main question is how to deal with outliers (this does not affect the default method 7, which finds the sample quantiles and not a population estimate). Wikipedia says this: Packages differ in how they estimate quantiles beyond the lowest and highest values in the sample, i.e. p < 1/N and p > (N − 1)/N. Choices include returning an error value, computing linear extrapolation, or assuming a constant value. The current choice is clipping (assuming a constant value), but this could be modified. Any feedback is appreciated! Otherwise, this will probably move forward in the current state for the next release. Cheers, Sebastian

8 8

Consistency of random number functions APIs.
by Todd Anderson Nov. 1, 2021

Nov. 1, 2021

Some NumPy random number generation functions take a dtype parameter whereas others don't. Some of them take an out parameter whereas others don't. Just glancing at it, there seems to be no rhyme or reason why this would be the case but is there some hidden consistency underneath the hood to explain why some have these params and others don't? Is there any reason that things like random.randn and numpy.random.Generator.normal don't take a dtype and out parameters? If I need to create a huge array of random numbers whose dtype is float16 or float32 then what is the BKM to do this when the routine I would like to use generates an array of float64 and with the 64-bit data type the array won't fit in memory?

3 2

Current state of the DType refactor
by Sebastian Berg Oct. 26, 2021

Oct. 26, 2021

Hi all, with the latest changes, NumPy has a completely revised infrastructure for: * Creating a new array from Python objects * Organizing casting (to align almost fully with ufuncs) * Organize the inner-loops and dispatching of ufuncs * Create and "promote" DTypes Which means that the core functionality of the DType NEPs is implemented: * https://numpy.org/neps/nep-0041-improved-dtype-support.html * https://numpy.org/neps/nep-0042-new-dtypes.html * https://numpy.org/neps/nep-0043-extensible-ufuncs.html This is an important milestone and it allows user-defined DTypes (and ufuncs) with functionality beyond what was previously possible. E.g. see the examples in: https://github.com/seberg/experimental_user_dtypes Current user-implemented dtypes should already be able to achieve better integration by moving towards the new system (experimentally!). Many dtypes (categoricals, units, datetimes, etc.) that were previously not possible can now be implemented as well! Future Work ----------- The work is by no means complete, but the quality of the next steps should change: from a focus on large and careful refactors towards small changes that often unlock new features for user DTypes! It also means that "feature requests" coming from users testing out the new boundaries would be extremely helpful! The new API pushes boundaries and we have to map out what this means :). One larger remaining change is that we should slowly modernize NumPy's own ufuncs to utilize the new API within NumPy itself; though this is not necessarily urgent. Here is a list of things of various importance and urgency: * Moving "legacy" implementations of certain `PyArray_ArrayFuncs` (https://numpy.org/devdocs/reference/c-api/types-and-structures.html#c.PyArr…) slots to a new API on the DType. This can be done incrementally, and we can ask users to keep using the old API and transition to the new API when it is ready. Some examples are `nonzero`, sorting functions, or `copyswapn` which are still used within NumPy. (NumPy should add new API for them, and "fill" the slots with generic implementations where possible.) * Some user DTypes will want to use "references" (data being allocated for each element) which requires cleanup (e.g. `free`, `Py_DECREF`). There are two potential solutions: - Refactor the reference counting in NumPy (would be good in any case) - Add support for DTypes that use Python objects as their "storage" and reuse the current code. * Physical units will want to conveniently re-use existing NumPy ufunc loops (math functions). The general infrastructure supports this, but API needs to be added to permit it and make it easy. * There are many smaller things, for example: - Allow a user DType without a python scalar (similar to astropy's Quantity), for which `arr1d[0]` returns a 0-D array. - A few tweaks to the current API (floating point errors and views) * We could now "fix" `dtype="S"` to mean a string with undefined length and reject `np.dtype("S")` but allow `np.array([1, 2], dtype="S")`. * I would also like to improve alignment tracking and handling (which is interesting for the public and private UFunc and Casting API). Generally, the API needs to be finalized and exposed, since this is currently only experimental with certain changes expected: https://github.com/numpy/numpy/blob/main/numpy/core/include/numpy/experimen… Value-based casting/promotion ----------------------------- One major difficulty (and cause of inconsistencies), is the use of value-based casting: arr = np.array([1, 2, 3, 4], dtype=np.uint8) arr + 5 # result is `uint8` arr - 255 # result is `int16` We do not wish to support this for user DTypes, but we may want to support "weak promotion", where the user DType knows that the other operand was a Python integer, float, or complex and can "downpromote" it if desired. In the above example, the first would still work and the second should error or warn. To some degree, this is an extension of existing logic and it is mostly implemented. However, it is not yet integrated into the ufuncs for use with user DTypes. It would be helpful to change NumPy's behavior, but that would likely be a major version change.

1 0

NumPy Community Meeting Wednesday
by Sebastian Berg Oct. 26, 2021

Oct. 26, 2021

Hi all, There will be a NumPy Community meeting Wednesday October 27th at 16:30 UTC. Everyone is invited and encouraged to join in and edit the work-in-progress meeting topics and notes at: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Best wishes Sebastian

1 0

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg Oct. 25, 2021

Oct. 25, 2021

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, October 20th at 16:30 UTC (9:30am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. Best regards Sebastian

3 2

old NumPy logo on partner websites
by Inessa Pawson Oct. 22, 2021

Oct. 22, 2021

I'm finding the old NumPy logo still in use on some partner websites. If you come across it, leave a comment in this issue https://github.com/numpy/numpy.org/issues/438. Please include a link and screenshot. I will follow up to get it updated. -- Every good wish, *Inessa Pawson*

3 2

4 Steps for writing an error-free academic essay
by thomaswhitetw773＠gmail.com Oct. 22, 2021

Oct. 22, 2021

Delivering an error-free essay is not an easy task as you will have to implement many essentialities before proceeding further. However, for a perfect essay report, you can avail help through different online platforms and easily hire experts for genuine guidance to write an essay. Related: [url=https://myassignmenthelp.com/au/expert-essay-assignment-writers-help-sy… help In Sydney[/url] However, one must never overlook essay writing as it is an integral part of academics and will help achieve desirable grades. But we can’t ignore errors; for sure, some minor-to-minor mistakes will prevail without any information. Through this blog, you can learn about the tips that will deliver appropriate measures to write a perfect essay report. [b]1. Analyzing essay topic[/b] Are you aware of the fact that you should understand a topic before writing your essay? If not, then ensure to study well your essay topic several times before approaching further. Then, with all possible methodologies, learn your essay question well or take help from the online platforms or other mediums for perfect essay writing. Finally, if you accumulate doubt to understand your topic, look for genuine help from your supervisor or online professionals. Related: [url=https://myassignmenthelp.com/essay-help/]Essay Help Australia[/url] [b]2. Follow structure for writing[/b] It is highly recommendable to work on the structure of essays for proper understanding and create informative content. It will introduce a well-structured essay quickly with all logical information by following an organized manner. Moreover, working on a structure before writing will help you visualize your essay report and work with suitable headings and sub-headings. [b]3. Carefully plan paragraphs[/b] Paragraphs play an essential role in essay writing. Each section brings focus on different information by following a similar concept of an essay topic. However, you can’t go beyond track while writing an essay; you must work on each paragraph to keep essentiality intact with appropriate measures. With a careful plan, you will never land with the question, “who will write my perfect essay?” As you are aware of essay topic requirements. Related: [url=https://myassignmenthelp.com/essay-help/essay-writers.html]Essay Writers Online[/url] [b]4. Proofreading and editing[/b] To deliver a flawless essay, you must make sure to conduct proofreading and editing. These will eliminate all kinds of mistakes, including grammatical errors entangling your document. Thus, proofreading before submission will enhance the quality of your essay report and make it sound credible. Accordingly, you can approach online experts for perfect essay help round the clock and conduct good proofreading. Along with the above steps, you can also approach professional experts for perfect essay writing help through an online website from the UK and other global regions. With a single click, you can hire an authentic service provider for a qualitative essay report. Related: [url=https://myassignmenthelp.com/write-my-essay-for-cheap.html]write my essay for cheap[/url] [b]Source:[/b] [url=https://nabur.montrosepress.com/user/thomas_white]https://nabur.montros…]

2 1

Flats In Bahria Town karachi
by invituscompany＠gmail.com Oct. 21, 2021

Oct. 21, 2021

Incredible 3 Rooms, spacious Apartments with a modern Design Kitchen, Reception Area, Standby Generator and Security Cameras. High Speed Lifts In Ideal Location, opposite Theme Park, Main Jinnah Avenue. Invictus provide wide range of apartment in <a href="https://invictus.com.pk/">Bahria Town Karachi</a>.

1 0

Moving triage meeting to the same slot as the community meeting
by Sebastian Berg Oct. 21, 2021

Oct. 21, 2021

Hi all, for simplicity, I suggest to move the triage meeting also by 30 minutes later to 16:30 UTC, sorry for not thinking of that when moving the community meeting. If no one protests I will announce it and change the calendar accordingly tomorrow. Cheers, Sebastian

2 1