Mailman 3 November 2020 - NumPy-Discussion

How did Numpy get its latest version of the documentation to appear at the top of Google search results?
by efremdan1 Nov. 13, 2020

Nov. 13, 2020

I'm working with Bokeh (https://docs.bokeh.org/en/latest/), another open-source Python package. The developers would like to have the latest version of their documentation appear at the top of Google search results when users search for information, but knowledge of how to do this is lacking. I've noticed that Numpy seems to have gotten this problem figured out, e.g., googling "numpy interpolate" results in the first hit being https://numpy.org/doc/stable/reference/generated/numpy.interp.html. This is unlike Python itself, where googling "python string formatting" results in the first hit being https://docs.python.org/3.4/library/string.html. So apparently someone in the Numpy developer world knows how to setup the doc pages in a manner that allows for this. Would that person be willing to post to the Bokeh message board on the topic (https://discourse.bokeh.org/t/some-unsolicited-feedback/6643/17) with some advice? Thank you! -- Sent from: http://numpy-discussion.10968.n7.nabble.com/

4 3

Re: [Numpy-discussion] Proposal: add the timestamp64 type
by Stefano Miccoli Nov. 13, 2020

Nov. 13, 2020

Discussion on time is endless! (Sorry for the extra noise, on the mailing list, but I would clarify some points.) If I got it right, np.datetime64 is defined by these points. 1) Internal representation: 64bit signed integer *plus* a time unit. The time unit can be expressed as - a SI valid unit (SI second and all decimal subunits up to the attosecond) - a SI acceptable unit (minute, hour, day) - a date unit (week, month, year) 2) Conversion routines: a bijective map from the internal representation to a proleptic Gregorian calendar [0] assuming a fixed epoch of 1970-01-01T00:00Z. The mapping neglects leap seconds and is not time-zone aware. I think that the current choice of 2) is a sensible one: I agree with Dan that it is useful to a wide audience, easy to compute, not ambiguous. I would discourage any attempt to implement in numpy more complex mappings, which are aware of time-zones and leap seconds, and why not, of the wide array of other time scales and time representation in use: this is a very complex task, and a nightmare from the point of view of maintenance. Other specialised libraries exist, like astropy.time [1] or dateutil [2] to this purpose. However the docs of numpy.datetime64 should be updated, to explicitly mention the use of the proleptic Gregorian calendar, and better clarify how the date units (month, year) are handled when casted to other shorter units like seconds, etc. Stefano [0] https://en.wikipedia.org/wiki/Proleptic_Gregorian_calendar [1] https://docs.astropy.org/en/stable/time/ [2] https://dateutil.readthedocs.io/en/stable/

1 0

API, NEP: Inclusion of the experimental `like=` argument in NumPy 1.20 (we currently lean to yes)
by Sebastian Berg Nov. 12, 2020

Nov. 12, 2020

Hi all, TL;DR: Should NumPy add a `like=` to array creation functions? This is an extension of the `__array_function__` protocol useful when working with array-like objects other than NumPy arrays. Including it, effectively means we preliminarily accept NEP 35. Note that without any feedback here, the current default is to include it in the upcoming NumPy 1.20 release. Long Version: Users who only work with NumPy arrays and no alternative array objects are not affected by this (but will see a "useless" keyword argument). However, dask and cupy, asked for the addition of a `like=` keyword argument to array creation functions (list below at [1]) in the proposed NEP 35: https://numpy.org/neps/nep-0035-array-creation-dispatch-with-array-function… This is an extension of the `__array_function__` protocol. I will refer to the well written NEP for details. It is important to note that there are alternative ideas under consideration [2]. This means there is a chance that the `like=` argument will be superseded by a different solution. My very personal angle currently is this: * Dask/CuPy have shown that this is useful to them * It does not seem like a big burden to me (aside from the new API/documentation which users might get confused by) * We could deprecate it again, even if it may be a slow process. With a lack of a strong argument against it and no clarity when alternatives might become available, I am fine with accepting it into NumPy [3]. However, I can certainly be swayed if anyone has concerns. There are also currently a few small outstanding discussion listed at: https://github.com/numpy/numpy/issues/17075 which is probably too technical for the decision whether or not to include it, but if you are interested in using the feature, lets discuss these as well! Cheers, Sebastian [1] The functions which receive the keyword argument are: * np.array, np.asarray, np.ascontiguousarray, etc. * np.arange * np.ones, np.zeros, np.empty, np.full * np.fromfunction * np.identity * np.fromfile * ... and a few I forgot ... [2] E.g. https://numpy.org/neps/nep-0037-array-module.html [3] There is the "middle ground": We could require an environment variable to activate it. But we discussed it briefly at the community meeting as well, and I think the consensus was there is probably no good argument for that. (e.g. it would mean the argument doesn't show up in the online documentation.)

2 1

Re: [Numpy-discussion] Proposal: add the timestamp64 type (Noam Yorav-Raphael)
by Stefano Miccoli Nov. 12, 2020

Nov. 12, 2020

On 11 Nov 2020, at 18:00, numpy-discussion-request(a)python.org<mailto:numpy-discussion-request@python.org> wrote: I propose to add a new type called "timestamp64". It will be a pure timestamp, meaning that it represents a moment in time (as seconds/ms/us/ns since the epoch), without any timezone information. Sorry, but I really don see the usefulness for another time stamping format based on POSIX time. Indeed POSIX time is based on a naive approximation of UTC and is ambiguous across leap seconds. Quoting from Wikipedia <https://en.wikipedia.org/wiki/Unix_time#Leap_seconds> The Unix time number 1483142400 is thus ambiguous: it can refer either to start of the leap second (2016-12-31 23:59:60) or the end of it, one second later (2017-01-01 00:00:00). In the theoretical case when a negative leap second occurs, no ambiguity is caused, but instead there is a range of Unix time numbers that do not refer to any point in UTC time at all. Precision time stamping is quite a complex task: you can use UTC, TAI, GPS, just to mention the most used timescales. And how do you deal with timestamps in the past, when timekeeping was based on earth rotation, and not atomic clocks ticking at (approximately) 1 SI-second frequency? In my opinion time-stamping should be application dependent, and I doubt that the new “timestamp64” could be beneficial to the numpy community. Best regards, Stefano

4 3

Proposal: add the timestamp64 type
by Noam Yorav-Raphael Nov. 11, 2020

Nov. 11, 2020

Hi, (I'm repeating things I wrote under the "datetime64: Remove deprecation warning..." thread, since I'm now proposing a new solution.) I propose to add a new type called "timestamp64". It will be a pure timestamp, meaning that it represents a moment in time (as seconds/ms/us/ns since the epoch), without any timezone information. It will have the exact same behavior as datetime64 had before version 1.11, except that its only allowed units will be seconds, milliseconds, microseconds and nanoseconds. Removing the longer units will make it clear that it doesn't deal with calendar and dates. Also, all the business day functionality will not be applicable to timestamp64. In order to get calendar information (such as the year) from timestamp64, you will have to manually convert it to python's datetime (or perhaps to np.datetime64) with an explicit timezone (utc, local, an offset, or a timezone object). This is needed because since the change introduced in 1.11, datetime64 no longer represents a timestamp, but rather a date and time of an abstract calendar. So given a datetime64, it is not possible to get an actual timestamp without knowing the timezone to which the datetime64 refers. If the datetime64 is in a timezone with daylight saving time, it can even be ambiguous, since the same written hour will occur twice on the transition from DST to winter time. I would like it to work like this: >>> np.timestamp64.now() numpy.timestamp64('2020-11-07 22:42:52.871159+0200') >>> np.timestamp64.now('s') numpy.timestamp64('2020-11-07 22:42:52+0200') >>> np.timestamp64(1604781916, 's') numpy.timestamp64('2020-11-07 22:42:52+0200') >>> np.timestamp64('2020-11-07 20:42:52Z') numpy.timestamp64('2020-11-07 22:42:52+0200') * timestamp64.now() will get an optional string argument with the base units. If not given, I think 'us' is a good default. * The repr will format the timestamp using the environment's timezone. * I like the repr to not include a 'T' between the date and the time. I find it much easier to read. * I tend to think that it should be allowed to construct a timestamp64 from an ISO8601 string without a timezone offset, in which case the environment's timezone will be used to convert it to a timestamp. So in the Asia/Jerusalem timezone it will look like: >>> np.timestamp64('2020-11-07 22:42:52') numpy.timestamp64('2020-11-07 22:42:52+0200') >>> np.timestamp64('2020-08-01 22:00:00') numpy.timestamp64('2020-08-01 22:00:00+0300') If I implement this, could it be added to numpy? Thanks, Noam

1 1

NumPy Community Meeting Wednesday
by Sebastian Berg Nov. 10, 2020

Nov. 10, 2020

Hi all, There will be a NumPy Community meeting Wednesday November 11th at 1pm Pacific Time (20:00 UTC). Everyone is invited and encouraged to join in and edit the work-in-progress meeting topics and notes at: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Best wishes Sebastian

1 0

Python 3.6 has been dropped from NumPy 1.20
by Charles R Harris Nov. 10, 2020

Nov. 10, 2020

Hi All, The subject says it all: Python 3.6 has been dropped for the NumPy 1.20 release. Chuck

2 1

ANN: SciPy 1.5.4
by Tyler Reddy Nov. 4, 2020

Nov. 4, 2020

Hi all, On behalf of the SciPy development team I'm pleased to announce the release of SciPy 1.5.4, which is a bug fix release that includes Python 3.9 wheels and a more complete fix for build issues on XCode 12. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.5.4 One of a few ways to install this release with pip: pip install scipy==1.5.4 ===================== SciPy 1.5.4 Release Notes ===================== SciPy 1.5.4 is a bug-fix release with no new features compared to 1.5.3. Importantly, wheels are now available for Python 3.9 and a more complete fix has been applied for issues building with XCode 12. Authors ====== * Peter Bell * CJ Carey * Andrew McCluskey + * Andrew Nelson * Tyler Reddy * Eli Rykoff + * Ian Thomas + A total of 7 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.5.4 ------------------------------- * `#12763 <https://github.com/scipy/scipy/issues/12763>`__: ndimage.fourier_ellipsoid segmentation fault * `#12789 <https://github.com/scipy/scipy/issues/12789>`__: TestConvolve2d.test_large_array failing on Windows ILP64 CI job * `#12857 <https://github.com/scipy/scipy/issues/12857>`__: sparse A[0,:] = ndarray is ok, A[:,0] = ndarray ValueError from... * `#12860 <https://github.com/scipy/scipy/issues/12860>`__: BUG: Build failure with Xcode 12 * `#12935 <https://github.com/scipy/scipy/issues/12935>`__: Failure to build with Python 3.9.0 on macOS * `#12966 <https://github.com/scipy/scipy/issues/12966>`__: MAINT: lint_diff.py on some backport PRs * `#12988 <https://github.com/scipy/scipy/issues/12988>`__: BUG: Highly multi-dimensional \`gaussian_kde\` giving \`-inf\`... Pull requests for 1.5.4 ------------------------------ * `#12790 <https://github.com/scipy/scipy/pull/12790>`__: TST: Skip TestConvolve2d.test_large_array if not enough memory * `#12851 <https://github.com/scipy/scipy/pull/12851>`__: BUG: sparse: fix inner indexed assignment of a 1d array * `#12875 <https://github.com/scipy/scipy/pull/12875>`__: BUG: segfault in ndimage.fourier_ellipsoid with length-1 dims * `#12937 <https://github.com/scipy/scipy/pull/12937>`__: CI: macOS3.9 testing * `#12957 <https://github.com/scipy/scipy/pull/12957>`__: MAINT: fixes XCode 12/ python 3.9.0 build for 1.5.x maint branch * `#12959 <https://github.com/scipy/scipy/pull/12959>`__: CI: add Windows Python 3.9 to CI * `#12974 <https://github.com/scipy/scipy/pull/12974>`__: MAINT: Run lint_diff.py against the merge target and only for... * `#12978 <https://github.com/scipy/scipy/pull/12978>`__: DOC: next_fast_len output doesn't match docstring * `#12979 <https://github.com/scipy/scipy/pull/12979>`__: BUG: fft.next_fast_len should accept keyword arguments * `#12989 <https://github.com/scipy/scipy/pull/12989>`__: BUG: improved the stability of kde for highly (1000s) multi-dimension... * `#13017 <https://github.com/scipy/scipy/pull/13017>`__: BUG: Add explicit cast to _tmp sum. * `#13022 <https://github.com/scipy/scipy/pull/13022>`__: TST: xfail test_maxiter_worsening() Checksums ========= MD5 ~~~ 09a446e10033c3132f1f257e3f4d9735 scipy-1.5.4-cp36-cp36m-macosx_10_9_x86_64.whl 25e58fde2fd4eb6c7717719db85e368b scipy-1.5.4-cp36-cp36m-manylinux1_i686.whl 2c9705cd57788ad79ea0c1015208f41f scipy-1.5.4-cp36-cp36m-manylinux1_x86_64.whl d0fb84f3ff45e4149698fbc662ac4d47 scipy-1.5.4-cp36-cp36m-manylinux2014_aarch64.whl f94f0e274cd2960ecb2d8751632e098c scipy-1.5.4-cp36-cp36m-win32.whl f56f4d5b67fccc49fb64331c28bdf7d1 scipy-1.5.4-cp36-cp36m-win_amd64.whl 33e0843f8619b78547866579134a733b scipy-1.5.4-cp37-cp37m-macosx_10_9_x86_64.whl 6720a406d82bd08c4370b665d5eddeb9 scipy-1.5.4-cp37-cp37m-manylinux1_i686.whl eafc3bc8a12d41cb348c73b54ad25ad5 scipy-1.5.4-cp37-cp37m-manylinux1_x86_64.whl 1174418ae0614d621acdb49faeaadcb8 scipy-1.5.4-cp37-cp37m-manylinux2014_aarch64.whl 5ca53c5cd6828498c0a41c3ae747a34b scipy-1.5.4-cp37-cp37m-win32.whl cdb91a7db9cf79b7446680f8d106aabc scipy-1.5.4-cp37-cp37m-win_amd64.whl 02a29a4eec9c61c30aef7439138fe1b3 scipy-1.5.4-cp38-cp38-macosx_10_9_x86_64.whl ce8e02167763493374c4bea807139a1b scipy-1.5.4-cp38-cp38-manylinux1_i686.whl 65ec027bfa6bed805dac62744b45c693 scipy-1.5.4-cp38-cp38-manylinux1_x86_64.whl c358b4b332cc9dbcd1eadc229d8b019e scipy-1.5.4-cp38-cp38-manylinux2014_aarch64.whl 492ec3bfe082229076a83d74cfa51d7e scipy-1.5.4-cp38-cp38-win32.whl d5d12211502429f3bc3074b12ca1f541 scipy-1.5.4-cp38-cp38-win_amd64.whl da25e7ac777e8b1b6cd7f117f163e6d2 scipy-1.5.4-cp39-cp39-macosx_10_9_x86_64.whl 12275e3578eb17065081d83d329d18db scipy-1.5.4-cp39-cp39-manylinux1_i686.whl 6778d670f75f536921c3d38e44517280 scipy-1.5.4-cp39-cp39-manylinux1_x86_64.whl efda61c74b29ffe714b6b842ec369a19 scipy-1.5.4-cp39-cp39-manylinux2014_aarch64.whl 107204c14328df879c5fc941e7829389 scipy-1.5.4-cp39-cp39-win32.whl ed6970f7538d38dd91a42950bd6843b7 scipy-1.5.4-cp39-cp39-win_amd64.whl 293401ee7ac354a2f2313373b497f40e scipy-1.5.4.tar.gz d446ec7a6b0bc44484389ab7589eccf5 scipy-1.5.4.tar.xz 47d0dabdc684475bc2aac7e8db9eea6f scipy-1.5.4.zip SHA256 ~~~~~~ 4f12d13ffbc16e988fa40809cbbd7a8b45bc05ff6ea0ba8e3e41f6f4db3a9e47 scipy-1.5.4-cp36-cp36m-macosx_10_9_x86_64.whl a254b98dbcc744c723a838c03b74a8a34c0558c9ac5c86d5561703362231107d scipy-1.5.4-cp36-cp36m-manylinux1_i686.whl 368c0f69f93186309e1b4beb8e26d51dd6f5010b79264c0f1e9ca00cd92ea8c9 scipy-1.5.4-cp36-cp36m-manylinux1_x86_64.whl 4598cf03136067000855d6b44d7a1f4f46994164bcd450fb2c3d481afc25dd06 scipy-1.5.4-cp36-cp36m-manylinux2014_aarch64.whl e98d49a5717369d8241d6cf33ecb0ca72deee392414118198a8e5b4c35c56340 scipy-1.5.4-cp36-cp36m-win32.whl 65923bc3809524e46fb7eb4d6346552cbb6a1ffc41be748535aa502a2e3d3389 scipy-1.5.4-cp36-cp36m-win_amd64.whl 9ad4fcddcbf5dc67619379782e6aeef41218a79e17979aaed01ed099876c0e62 scipy-1.5.4-cp37-cp37m-macosx_10_9_x86_64.whl f87b39f4d69cf7d7529d7b1098cb712033b17ea7714aed831b95628f483fd012 scipy-1.5.4-cp37-cp37m-manylinux1_i686.whl 25b241034215247481f53355e05f9e25462682b13bd9191359075682adcd9554 scipy-1.5.4-cp37-cp37m-manylinux1_x86_64.whl fa789583fc94a7689b45834453fec095245c7e69c58561dc159b5d5277057e4c scipy-1.5.4-cp37-cp37m-manylinux2014_aarch64.whl d6d25c41a009e3c6b7e757338948d0076ee1dd1770d1c09ec131f11946883c54 scipy-1.5.4-cp37-cp37m-win32.whl 2c872de0c69ed20fb1a9b9cf6f77298b04a26f0b8720a5457be08be254366c6e scipy-1.5.4-cp37-cp37m-win_amd64.whl e360cb2299028d0b0d0f65a5c5e51fc16a335f1603aa2357c25766c8dab56938 scipy-1.5.4-cp38-cp38-macosx_10_9_x86_64.whl 3397c129b479846d7eaa18f999369a24322d008fac0782e7828fa567358c36ce scipy-1.5.4-cp38-cp38-manylinux1_i686.whl 168c45c0c32e23f613db7c9e4e780bc61982d71dcd406ead746c7c7c2f2004ce scipy-1.5.4-cp38-cp38-manylinux1_x86_64.whl 213bc59191da2f479984ad4ec39406bf949a99aba70e9237b916ce7547b6ef42 scipy-1.5.4-cp38-cp38-manylinux2014_aarch64.whl 634568a3018bc16a83cda28d4f7aed0d803dd5618facb36e977e53b2df868443 scipy-1.5.4-cp38-cp38-win32.whl b03c4338d6d3d299e8ca494194c0ae4f611548da59e3c038813f1a43976cb437 scipy-1.5.4-cp38-cp38-win_amd64.whl 3d5db5d815370c28d938cf9b0809dade4acf7aba57eaf7ef733bfedc9b2474c4 scipy-1.5.4-cp39-cp39-macosx_10_9_x86_64.whl 6b0ceb23560f46dd236a8ad4378fc40bad1783e997604ba845e131d6c680963e scipy-1.5.4-cp39-cp39-manylinux1_i686.whl ed572470af2438b526ea574ff8f05e7f39b44ac37f712105e57fc4d53a6fb660 scipy-1.5.4-cp39-cp39-manylinux1_x86_64.whl 8c8d6ca19c8497344b810b0b0344f8375af5f6bb9c98bd42e33f747417ab3f57 scipy-1.5.4-cp39-cp39-manylinux2014_aarch64.whl d84cadd7d7998433334c99fa55bcba0d8b4aeff0edb123b2a1dfcface538e474 scipy-1.5.4-cp39-cp39-win32.whl cc1f78ebc982cd0602c9a7615d878396bec94908db67d4ecddca864d049112f2 scipy-1.5.4-cp39-cp39-win_amd64.whl 4a453d5e5689de62e5d38edf40af3f17560bfd63c9c5bd228c18c1f99afa155b scipy-1.5.4.tar.gz 5c87347bfe2db6e23d391aa226584f6b280248c0ca71e08f26f1faf9d7a76bc9 scipy-1.5.4.tar.xz e0bcc10c133a151937550bb42301c56439d34098b1b8f9dd18c5919d604edd37 scipy-1.5.4.zip

1 0

New package to speed up ufunc inner loops
by Matti Picus Nov. 4, 2020

Nov. 4, 2020

Hi. On behalf of Quansight and RTOSHoldings, I would like to introduce "pnumpy", a package to speed up NumPy. https://quansight.github.io/numpy-threading-extensions/stable/index.html What is in it? - use "PyUFunc_ReplaceLoopBySignature" to hook all the UFunc inner loops - When the inner loop is called with a large enough array, chunk the data and perform the iteration via a thread pool - Add a different memory allocator for "ndarray" data (will require an appropriate API from NumPy) - Allow using optimized loops above and beyond what NumPy provides - Allow logging inner loop calls and parameters to learn about the current process and perhaps tune the performance accordingly The first release contains the hooking mechanism and the thread pool, the rest has been prototyped but is not ready for release. The idea behind the package is that a third-party package can try things out and iterate much faster than NumPy. If some of the ideas bear fruit, and do not add an undue maintenance burden to NumPy, the code can be ported to NumPy. I am not sure NumPy wishes to take upon itself the burden of managing threads, but a third-party package may be able to. I am writing to the mailing list both to announce the pre-release under the wrong name, and, in accordance with the fair play rules[1], to request use of the "numpy" name in the package. We had considered many options, in the end would like to propose "pnumpy" (the p is either "parallel" or "performant" or "preliminary", whatever you desire). Matti [1] https://numpy.org/neps/nep-0036-fair-play.html#fair-play-rules

6 11

NumPy Development Meeting Wednesday - Triage Focus (US switched times last week)
by Sebastian Berg Nov. 4, 2020

Nov. 4, 2020

Hi all, Our bi-weekly triage-focused NumPy development meeting is today (Wednesday, November 4th) at 11 am Pacific Time (18:00 UTC). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. Best regards Sebastian

1 0