Mailman 3 December 2021 - NumPy-Discussion

Proposal for new function to determine if a float contains an integer
by Joseph Fox-Rabinovitz 31 Dec '21

31 Dec '21

Hi, I wrote a reference implementation for a C ufunc, `isint`, which returns True for integers and False for non-integers, found here: https://github.com/madphysicist/isint_ufunc. The idea came from a Stack Overflow question of mine, which has gotten a fair number of views and even some upvotes: https://stackoverflow.com/q/35042128/2988730. The current "recommended" solution is to use ``((x % 1) == 0)``. This is slower and more cumbersome because of the math operations and the temporary storage. My version returns a single array of booleans with no intermediaries, and is between 5 and 40 times faster, depending on the type and size of the input. If you are interested in taking a look, there is a suite of tests and a small benchmarking script that compares the ufunc against the modulo expression. The entire thing currently works with bit twiddling on an appropriately converted integer representation of the number. It assumes a standard IEEE754 representation for float16, float32, float64. The extended 80-bit float128 format gets some special treatment because of the explicit integer bit. Complex numbers are currently integers only if they are real and integral. Integer types (including bool) are always integers. Time and text raise TypeErrors, since their integerness is meaningless. If a consensus forms that this is something appropriate for numpy, I will need some pointers on how to package up C code properly. This was an opportunity for me to learn to write a basic ufunc. I am still a bit confused about where code like this would go, and how to harness numpy's code generation. I put comments in my .c and .h file showing how I would expect the generators to look, but I'm not sure where to plug something like that into numpy. It would also be nice to test on architectures that have something other than a 80-bit extended long double instead of a proper float128 quad-precision number. Please let me know your thoughts. Regards, - Joe

3 4

Re: NumPy-Discussion Digest, Vol 183, Issue 33
by Stefano Miccoli 31 Dec '21

31 Dec '21

Lev, excuse me if I go in super pedantic mode, but your answer and the current text of the article fail to grasp an important point. 1) The proleptic Gregorian calendar is about leap year rules. It tracks days without making any assumption on the length of days. If we agree on using this calendar, dates like -0099-07-12 and 2021-12-29 are defined without ambiguity, and we can easily compute the number of days between these two dates. 2) Posix semantics is about the length of a day, and is based on the (utterly wrong) assumption that a mean solar day is constant and exactly 86400 SI seconds long. (For an authoritative estimate of historical length of day variations see <http://astro.ukho.gov.uk/nao/lvm/ <http://astro.ukho.gov.uk/nao/lvm/>> and the related papers <http://doi.org/10.1098/rspa.2016.0404 <http://doi.org/10.1098/rspa.2016.0404> <https://doi.org/10.1098/rspa.2020.0776 <https://doi.org/10.1098/rspa.2020.0776>>) Knowing assumption 1) is important when coding dates before 1582-10-15: e.g. 1582-10-04 Julian is 1582-10-14 proleptic Gregorian. Once we agree on the proleptic Gregorian calendar everything works as expected: time deltas expressed in days are correct. Knowing assumption 2) is important if we pretend to compute time deltas for date-time objects with high precision: e.g. how many SI seconds occur between 1582-10-14T12:00:00 and 1582-10-15T12:00:00 with millisecond precision? Here we must first define what T12:00:00 means, say UT1, but most critically we need to know the length of day in 1582. With Posix semantics a day is always 86400.000 SI second long; however the real value of the length of day in 1582 could be about 5 ms less. The problem here is that small errors accumulate and if we compute the difference between 0000-01-01T12:00:00 and 1900-01-01T12:00:00 the numpy answer may be off by about 10_000 seconds. Fast forward to current times: after 1972 T12:00:00 should be defined as UTC, and the posix assumption is correct for almost every day, bar when a leap second is added (86401 s) or removed (86399 s, but this has never occurred.) Now the numpy computed timedeltas are correct up to an integral number of seconds that can be derived from a leap second table, if both dates are in the past. If one or both of the dates are in the future, then we must rely on models of earth rotation, and estimate the future introduction of leap seconds. But earth rotation is quite “unpredictable”, so usually this is not very accurate. The main problem with numpy datetime64 is that by using np.int64 for Datetimes it gives 1/2**63 precision (about 1e-19). But this apparent very high precision has to be confronted with the relative accuracy of the Posix semantics, which lies at about 1e-7, 1e-8, if we look at timespans of a couple of centuries. So I agree that the np.datetime64 precision is somehow misleading. This all said, proleptic Gregorian + Posix semantics is, in my opinion, the only sensible option in a numerical package like numpy, although the results can be inaccurate. However errors are usually small on the average (say 10 ms/day which is about 1e-7). Everything more sophisticated is in the realm of specialised packages, like AstroPy, but also Skyfield <https://rhodesmill.org/skyfield/ <https://rhodesmill.org/skyfield/>>. Stefano > On 28 Dec 2021, at 21:35, numpy-discussion-request(a)python.org wrote: > > t is not a matter of formal definitions. Leap seconds are uncompromisingly practical. > If you look at the wall clock on 1 Jan 1970 00:00 and then look at the same clock today and measure the difference with atomic clock you won't get the time delta that np.timedelta64 reports. There will be a difference of ~37 seconds. Actually this should be 27s. > One would expect that a library claiming to work with attoseconds would at least count the seconds correctly ) > Astropy library calculates <https://het.as.utexas.edu/HET/Software/Astropy-1.0/api/astropy.time.TimeGPS…> them properly: > "GPS Time. Seconds from 1980-01-06 00:00:00 UTC For example, 630720013.0 is midnight on January 1, 2000." > >>> np.datetime64('2000-01-01', 's') - np.datetime64('1980-01-06', 's') > numpy.timedelta64(630720000,'s') > > Everything should be made as simple as possible but not simpler. Leap seconds are an inherent part of the world we live in. > > Eg this is how people deal with them currently: they have to parse times like 23:59:60.209215 manually > https://stackoverflow.com/questions/21027639/python-datetime-not-accounting… <https://stackoverflow.com/questions/21027639/python-datetime-not-accounting…> > > - calendrical calculations are performed using a proleptic Gregorian calendar <https://en.wikipedia.org/wiki/Proleptic_Gregorian_calendar <https://en.wikipedia.org/wiki/Proleptic_Gregorian_calendar>>, > - Posix semantics is followed, i.e. each day comprises exactly 86400 SI seconds, thus ignoring the existence of leap seconds. > > I would also point out that this choice is consistent with python datetime. > But not consistent with python time ;) "Unlike the time module, the datetime module does not support leap seconds." > • time.CLOCK_TAI > International Atomic Time > The system must have a current leap second table in order for this to give the correct answer. PTP or NTP software can maintain a leap second table. > Availability: Linux. > New in version 3.9. > > As what regards the promised future support for leap seconds, I would not mention it, for now. In fact leap second support requires a leap second table, which is not available on all platforms supported by numpy. Therefore the leap second table should be bundled and updated with every numpy release with the very undesirable effect that older version (with outdated tables) would behave differently from newer ones. > The olson database is much larger yet it is updated on millions of computers, phones and what not without causing extra difficulties > (except when the government unexpectedly decides to shift a region from one TZ to another). This way developers have a choice whether > to work with naive datetimes (ok in a single timezone without daylight-saving) or with timezone-aware ones (and take care about updating the pytz). > > This is how astropy deals with updating the table: https://docs.astropy.org/en/stable/api/astropy.utils.iers.LeapSeconds.html <https://docs.astropy.org/en/stable/api/astropy.utils.iers.LeapSeconds.html> > Pytz also has this table both inside the binary tz files and in a text file: https://github.com/stub42/pytz/blob/master/tz/leap-seconds.list <https://github.com/stub42/pytz/blob/master/tz/leap-seconds.list> > which it in turn downloads from NIST ftp://ftp.nist.gov/pub/time/leap-seconds.list <ftp://ftp.nist.gov/pub/time/leap-seconds.list> > It is in the public domain, NIST updates this file regularly and it even has an expiration date (presently it is 28 June 2022). > Activation of the 'leap-second-aware mode' could be made dependent on the presence of the pytz mode and/or this expiration date. > > I don't think having a non-default leap-second-aware mode would hurt anyone, but I also wouldn't consider it a priority. I think when someone needs them he'll make a patch and until that moment it is safe to have them as 'proposed' ) > > I feel that leap seconds should be mentioned somewhere—in the article or in the docs, because it limits practical precise usage of timedelta64 to > a period between 2021 and 2016 (last time when a leap second was injected). A modest timespan for a library claiming to work with years upto 9.2e18 BC ;) > > Thank you for your suggestions! I've included them into the article, plz have a look at the updated version. > > Best regards, > Lev >

2 1

Who is the tallest and strongest Chinese basketball player?
by Just allstar 31 Dec '21

31 Dec '21

Yao Ming is considered to be the wonderful as well as a best chinese basketball player. He is an executive as well as a professional <a href="https://www.justallstar.com/20-fun-facts-about-yao-ming-and-how-he-re-shape…">chinese basketball player</a>. Yao ming was born in shanghai and he is considered to be a very powerful as well as a professional among all the other players of the national basketball association team. He also won a national basketball championship in the final year. Yao ming has a height of 6 feets and 3 inches.

2 1

An article on numpy data types
by Lev Maximov 30 Dec '21

30 Dec '21

Hi everyone, I'm almost done with the article about numpy types – something I haven't covered in Numpy Illustrated. Would someone please have a look to confirm I haven't written anything anti-climatic there? https://axil.github.io/numpy-data-types.html -- Best regards, Lev PS Earlier today I've mistakenly sent an email with the wrong link.

10 26

Call for ideas: 2021 user survey report
by Stephanie Mendoza 29 Dec '21

29 Dec '21

The NumPy survey team is seeking ideas from the community about potential softwares or tools that we could use to generate a report of insights for the 2021 user survey. In 2020, we used Google Charts and Adobe Illustrator to create the following report: https://numpy.org/surveys/NumPy_usersurvey_2020_report.pdf. This year, we are in search of a new tool that will require less manual effort, and is somewhat reproducible for creating survey reports in future years. Ideally, this tool will allow us to create a report that is as visually appealing as the 2020 report. An open-source tool is preferred but not required. If you have any ideas or thoughts that may help the survey team, please respond via the mailing list. -Stephanie Mendoza

1 0

ANN: SfePy 2021.4
by Robert Cimrman 28 Dec '21

28 Dec '21

I am pleased to announce the release of SfePy 2021.4. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by finite element methods. It is distributed under the new BSD license. Home page: https://sfepy.org Mailing list: https://mail.python.org/mm3/mailman3/lists/sfepy.python.org/ Git (source) repository, issue tracker: https://github.com/sfepy/sfepy Highlights of this release -------------------------- - improved pyvista-based visualization script resview.py - gallery images generated using resview.py - homogenization tools: new parallel recovery of multiple microstructures - new "dry water" flow example For full release notes see [1]. Cheers, Robert Cimrman [1] http://docs.sfepy.org/doc/release_notes.html#id1 --- Contributors to this release in alphabetical order: Robert Cimrman Yves Delley Florian Le Bourdais Vladimir Lukes Brylie Christopher Oxley Tomáš Zítka

1 0

Re: An article on numpy data types
by Stefano Miccoli 28 Dec '21

28 Dec '21

Nice overview! In my opinion section 5. Datetimes could benefit from some clarifications, and contains a couple of typos. Let me start with the typos. I read "As in pure python, np.datetime64 is accompained by np.timedelta64 (stored as a single np.uint64) with the expectable arithmetic operators.” but it should be “np.int64” since time delta values are signed. "Leap seconds are supported: <code snippet> Leap seconds are not (but proposed) <code snippet>" Of course the first sentence should be “leap years”, which leads to my main point. It makes no sense to claim “leap year support” without specifying the relevant calendar. Thus I would suggest to clearly state, from the very begin of this section, that - calendrical calculations are performed using a proleptic Gregorian calendar <https://en.wikipedia.org/wiki/Proleptic_Gregorian_calendar <https://en.wikipedia.org/wiki/Proleptic_Gregorian_calendar>>, - Posix semantics is followed, i.e. each day comprises exactly 86400 SI seconds, thus ignoring the existence of leap seconds. I would also point out that this choice is consistent with python datetime. As what regards the promised future support for leap seconds, I would not mention it, for now. In fact leap second support requires a leap second table, which is not available on all platforms supported by numpy. Therefore the leap second table should be bundled and updated with every numpy release with the very undesirable effect that older version (with outdated tables) would behave differently from newer ones. Stefano

2 1

help with PyArrayInterface
by bloring＠lbl.gov 27 Dec '21

27 Dec '21

I would like to make a C++ data structure that internally has a contiguous buffer look like a Numpy array via the numpy ArrayInterface. I already have this structure wrapped and usable in Python and things seem fairly straight forward except I am not sure about the typekind field of the PyArrayInterface structure. In the documentation (here https://numpy.org/doc/stable/reference/c-api/types-and-structures.html#c.Py…) I see that typekind is a char and that there are values for some but not all of the various C++ POD numeric types.. What typekind value should I use for single precision floating point, 8 bit ineteger, 16 bit integer, etc etc? Those types are all supported by Numpy, but I don't know what the typekind value should be. Can you point to a list of vlaues?

2 2

New
by Isha Verma 27 Dec '21

27 Dec '21

Hello I am new to NumPy and I would like to contribute. Can someone please help me with the ways I can contribute !

2 1

hello from your Contributor Experience Lead
by Inessa Pawson 27 Dec '21

27 Dec '21

Hi, everyone! Most of you know me as an active contributor on the NumPy web and survey teams and organizer of numerous Python community events. At the beginning of this month, I started in a new role at NumPy – a Contributor Experience Lead. Stop by and say hi at the next NumPy Newcomers' Hour which is scheduled for *today, Dec 16th at 4 pm UTC**. Follow the link to join via Zoom: https://zoom.us/j/6345425936 *To convert the start time to the local time of the area you are in, use the following link: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Newcomers%2… -- Every good wish, Inessa

2 1