Mailman 3 January 2022 - NumPy-Discussion

NumPy Community Meeting Wednesday
by Sebastian Berg Jan. 4, 2022

Jan. 4, 2022

Hi all, There will be a NumPy Community meeting Wednesday January 5th at 19:00 UTC. Everyone is invited and encouraged to join in and edit the work-in-progress meeting topics and notes at: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Best wishes Sebastian

1 0

MyGrad 2.1.0 has been released
by rsoklaski＠gmail.com Jan. 3, 2022

Jan. 3, 2022

Hi all, I am pleased to announce that MyGrad 2.1.0 has been released. In accordance with NEP 29, we have dropped support for NumPy < 1.19. Changelog: https://mygrad.readthedocs.io/en/latest/changes.html#v2-1-0 What the heck is MyGrad? MyGrad provides drop-in automatic differentiation to native NumPy code. Just pass MyGrad-tensors into your NumPy-based code (instead of arrays), and start differentiating! It accomplishes this by making keen use of NumPy's function-override mechanisms.

1 0

Re: An article on numpy data types (Lev Maximov)
by Stefano Miccoli Jan. 1, 2022

Jan. 1, 2022

First of all, happy new 2022 UTC year! Let my add just a very brief note to this discussion: I opened https://github.com/numpy/numpy/issues/20675 <https://github.com/numpy/numpy/issues/20675> which addresses the shortcomings of the current documentation, which is in my opinion not sufficiently explicit in stating the datetime64 semantics. It is true that these are largely consistent with python ‘datetime.datetime’, but ‘explicit is better than implicit’. If nobody objects I will then open a doc-only PR adding a very short paragraph to the docs trying to explain the points discussed here. As what regards how much time UTC gained from 1970-01-01 up to today, you are right, it’s about 29 s. The UTC timescale was officially born in 1963 but it can be traced back at least up to 1956/1958, see https://github.com/skyfielders/python-skyfield/issues/679 <https://github.com/skyfielders/python-skyfield/issues/679> where this is discussed with reference to the timescales implemented in python-skyfield. Stefano > On 31 Dec 2021, at 13:27, numpy-discussion-request(a)python.org wrote: > > Hey, Stefano! > > The level of being pedantic is absolutely acceptable. > > I don't question any of your arguments. They are all perfectly valid. > > Except that I'd rather say it is ~29 seconds if measuring against 1970. Leap seconds were introduced in 1972 and there were > a total of 27 seconds since then, but TAI time was ticking since 1958 and gained 10 seconds by 1970 so it is approximately 0.83 second per year at which gives approx 28.67 sec between today and 1970. > So 1970 is a bad choice of epoch if you want to introduce a leap-second-aware datetime. > In GPS time they chose 1980. In TAI it is 1958, but that is somewhat worse than 1980 because it is not immediately clear how to perform the conversion timestamp<->timedelta between 1958 and 1970. > > Something like 'proleptic gps time' would be needed to estimate the number of leap seconds in the years before 1972 when they were introduced. Or maybe to limit the leap-second timescale > to start at 1972 and not to accept any timestamps before that date. > > The system that ignores the existence of the leap seconds has a right to exist. > But it just has limited applicability. > > np.datetime64 keeps time as a delta between the moment in time and a predefined epoch. > Which standard does it use to translate this delta to human-readable time in years, > months, and so on? > > If it is UTC, then it must handle times like 2016-12-31 23:59:60, because it is a valid UTC timestamp. > >>> np.datetime64('2016-12-31 12:59:60') > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > ValueError: Seconds out of range in datetime string "2016-12-31 12:59:60" > > Datetime also fails (so far) to handle it: > >>> dt(2016,12,31,23,59,60) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > ValueError: second must be in 0..59 > > But `time` works. Well, at least it doesn't raise an exception: > >>> t = time.struct_time((2016,12,31,12,59,60,0,0,0)); t > time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=12, tm_min=59, tm_sec=60, tm_wday=0, tm_yday=0, tm_isdst=0) > >>> time.asctime(t) > 'Mon Dec 31 12:59:60 2016' > >>> time.gmtime(calendar.timegm(t)) > time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0, tm_sec=0, tm_wday=6, tm_yday=1, tm_isdst=0) > > Imagine a user that decides which library to use to store some (life critical!) measurements taken every 100 ms. He looks at NumPy datetime64, reads that it is capable of handling attosecods, and decides that it is a perfect fit. Now imagine that on 31 Dec 2022 the World Government decided to inject a leap second. The system will receive the announcement from the NTC servers and will > prepare to replay this second twice. As soon as this moment chimes in he'll run into a ValueError, which he won't notice because he's celebrating a New Year :) And guess whom he'll blame? ;) > > Actually the humanity has already got used to replaying timespans twice. It happens every year in the countries that observe daylight saving time. And the solution is to use a more linear scale than local time, namely, UTC. But now turns out that UTC is not linear enough and it also has certain timespans happening twice. > > The solution once again is use a _really_ linear time which is TAI. I think python 'time' library did a right thing to introduce time.CLOCK_TAI, after all. > > Astropy handles the UTC scale properly though: > >>> t = Time('2016-12-31 23:59:60') > <Time object: scale='utc' format='iso' value=2016-12-31 23:59:60.000> > >>> t0 = Time('2016-12-31 23:59:59') > <Time object: scale='utc' format='iso' value=2016-12-31 23:59:59.000> > >>> delta = t-t0 > <TimeDelta object: scale='tai' format='jd' value=1.1574074074038876e-05> > >>> delta.sec > 0.9999999999969589 > >>> t0 + delta > <Time object: scale='utc' format='iso' value=2016-12-31 23:59:60.000> > > So the solution for that particular person with regular intervals of time is to use astropy. I mention it in the article. > I made some corrections to the text. I'd be grateful if you had a look and pointed me to the particular sentences > that need improvement. > > Best regards, > Lev >

3 3

NumPy 1.22.0 has been released
by Charles R Harris Jan. 1, 2022

Jan. 1, 2022

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 1.22.0. NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are: - Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release. - A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across applications such as CuPy and JAX. - NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data. - New methods for `quantile`, `percentile`, and related functions. The new methods provide a complete set of the methods commonly found in the literature. - A new configurable allocator for use by downstream projects. These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation. The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.22.0/>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.22.0>. Linux users will need pip >= 0.19.3 in order to install the manylinux2014 wheels. A recent version of pip is needed to install the universal2 macos wheels. *Contributors* A total of 153 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - @DWesl - @Illviljan - @h-vetinari - @yan-wyb + - Aaron Meurer - Abel Aoun + - Adrian Gao + - Ahmet Can Solak + - Ajay DS + - Alban Colley + - Alberto Rubiales + - Alessia Marcolini + - Amit Kumar + - Andrei Batomunkuev + - Andrew Watson + - Anirudh Dagar + - Ankit Dwivedi + - Antony Lee - Arfy Slowy + - Arryan Singh + - Arun Palaniappen + - Arushi Sharma + - Bas van Beek - Brent Brewington + - Carl Johnsen + - Carl Michal + - Charles Harris - Chiara Marmo - Chris Fu (傅立业) + - Christoph Buchner + - Christoph Reiter + - Chunlin Fang - Clément Robert + - Constanza Fierro - Damien Caliste - Daniel Ching - David Badnar + - David Cortes + - David Okpare + - Derek Huang + - Developer-Ecosystem-Engineering + - Dima Pasechnik - Dimitri Papadopoulos + - Dmitriy Fishman + - Eero Vaher + - Elias Koromilas + - Eliaz Bobadilla + - Elisha Hollander + - Eric Wieser - Eskild Eriksen + - Evan Miller + - Fayas Noushad + - Gagandeep Singh + - Ganesh Kathiresan - Ghiles Meddour + - Greg Lucas - Gregory R. Lee - Guo Shuai + - Gwyn Ciesla + - Hameer Abbasi - Hector Martin + - Henry Schreiner + - Himanshu + - Hood Chatham + - Hugo Defois + - Hugo van Kemenade - I-Shen Leong + - Imen Rajhi + - Irina Maria Mocan + - Irit Katriel + - Isuru Fernando - Jakob Jakobson - Jerry Morrison + - Jessi J Zhao + - Joe Marshall + - Johan von Forstner + - Jonas I. Liechti + - Jonathan Reichelt Gjertsen + - Joshua Himmens + - Jérome Eertmans - Jérôme Kieffer + - KIU Shueng Chuan + - Kazuki Sakamoto + - Kenichi Maehashi - Kenny Huynh + - Kent R. Spillner + - Kevin Granados + - Kevin Modzelewski + - Kevin Sheppard - Lalit Musmade + - Malik Idrees Hasan Khan + - Marco Aurelio da Costa + - Margret Pax + - Mars Lee + - Marten van Kerkwijk - Matthew Barber + - Matthew Brett - Matthias Bussonnier - Matthieu Dartiailh - Matti Picus - Melissa Weber Mendonça - Michael McCann + - Mike Jarvis + - Mike McCann + - Mike Toews - Mukulika Pahari - Nick Pope + - Nick Wogan + - Niels Dunnewind + - Niko Savola + - Nikola Forró - Niyas Sait + - Pamphile ROY - Paul Ganssle + - Pauli Virtanen - Pearu Peterson - Peter Hawkins + - Peter Tillema + - Prathmesh Shirsat + - Raghuveer Devulapalli - Ralf Gommers - Robert Kern - Rohit Goswami + - Ronan Lamy - Ross Barnowski - Roy Jacobson + - Samyak S Sarnayak + - Sayantika Banik + - Sayed Adel - Sebastian Berg - Sebastian Schleehauf + - Serge Guelton - Shriraj Hegde + - Shubham Gupta + - Sista Seetaram + - Stefan van der Walt - Stephannie Jimenez Gacha + - Tania Allard - Theodoros Nikolaou + - Thomas Green + - Thomas J. Fan - Thomas Li + - Tim Hoffmann - Tom Tan + - Tyler Reddy - Vijay Arora + - Vinith Kishore + - Warren Weckesser - Yang Hau - Yashasvi Misra - Yuval Ofek + - Zac Hatfield-Dodds - Zhang Na + Cheers, Charles Harris

1 0

Proposal to add method for string slicing to np.char
by Joseph Fox-Rabinovitz Jan. 1, 2022

Jan. 1, 2022

I made a PR for another new method `np.char.slice_` here: https://github.com/numpy/numpy/pull/20694 Here is an excerpt of the PR message: There are numerous examples of string slicing being a fairly requested feature: - https://stackoverflow.com/q/70547027/2988730 - https://stackoverflow.com/q/39042214/2988730 - https://stackoverflow.com/q/40976714/2988730 - https://stackoverflow.com/q/64981711/2988730 - https://stackoverflow.com/q/31387047/2988730 - https://stackoverflow.com/q/69856133/2988730 - ... I stopped searching around here Given the existence of the `char` module, there is no reason not to include a basic slicing operation that can work cheaper than making views and copies of a string, or switching to pandas for this one feature. This PR introduces such a function. It's written entirely in python, and does its absolute best not to make a copy of any data. The original inspiration for this is my answer to the first question in the list above. I've added a couple of features since then, like the ability to have a meaningful non-unit step and the ability to set the length of non-unit-step chunks. Please let me know your thoughts about the value of something like this. Regards, - Joe

1 0