Mailman 3 May 2019 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker July 11, 2023

July 11, 2023

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem Feb. 18, 2023

Feb. 18, 2023

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How … [View More]

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell July 22, 2021

July 22, 2021

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program … [View More]makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil [View Less]

10 10

scientific Python featured in GitHub keynote
by Ralf Gommers June 2, 2019

June 2, 2019

Hi all, On Thursday I had the pleasure to be at GitHub Satellite, together with quite a few other maintainers from projects throughout our ecosystem, and see NumPy, Matplotlib, AstroPy and other projects highlighted prominently in Nat Friedman's keynote. It included the story of the black hole image, and the open source software that enabled that image. It's the first 21 minutes of https://www.youtube.com/watch?v=xAbJkn4uRL4. Also, we now have "used by" for each repo and the dependency graph (… [View More]

2 3

Histgrom density estimation (density kwarg) with out of range values
by Sebastian Berg May 31, 2019

May 31, 2019

Hi all, unfortunately it was noticed in Issue 13604 [0] that when histogram is given used with a specified range and the `density=True` keyword argument out of bound values are simply discarded [1]. Discarding out of bound values makes sense when the density/normed option is not used, since in that case event counts are reported. However, when out of bound values exist, the probability density should arguably not sum up to 1 anymore. We seem to have three ways possible to continue here: 1. … [View More]

1 0

Community Call -- May 29/ 2019
by Tyler Reddy May 29, 2019

May 29, 2019

Hi, There will be a NumPy Community Call on May 29/ 2019 at 11 am Pacific Time. Anyone may join and edit the work-in-progress meeting notes: https://hackmd.io/Au2YB5QpQjyFUcCfdT1efw?view I think we still need to sort out a link / medium for the call--hopefully that will get added to the document in time. Best wishes, Tyler

1 1

NumPy 1.16.4 released.
by Charles R Harris May 28, 2019

May 28, 2019

Charles R Harris <charlesr.harris(a)gmail.com> Apr 21, 2019, 8:39 PM to numpy-discussion, SciPy, SciPy-User, bcc: python-announce-list Hi All, On behalf of the NumPy team I am pleased to announce the release of NumPy 1.16.4 which contains several fixes for newly reported bugs. The Python versions supported in this release are 2.7 and 3.5-3.7. Downstream developers building this release should use Cython >= 0.29.2 and, if using OpenBLAS, OpenBLAS > v0.3.7. Wheels for this release … [View More]can be downloaded from PyPI <https://pypi.org/project/numpy/1.16.4/>, source archives and release notes are available from Github <https://github.com/numpy/numpy/releases/tag/v1.16.4>. If you are installing using pip, you may encounter a problem with older installed versions of NumPy that pip did not delete becoming mixed with the current version, resulting in an ``ImportError``. That problem is particularly common on Debian derived distributions due to a modified pip. The fix is to make sure all previous NumPy versions installed by pip have been removed. See #12736 <https://github.com/numpy/numpy/issues/12736> for discussion of the issue. *Contributors* A total of 10 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Charles Harris - Eric Wieser - Dennis Zollo + - Hunter Damron + - Jingbei Li + - Kevin Sheppard - Matti Picus - Nicola Soranzo + - Sebastian Berg - Tyler Reddy *Pull requests merged* A total of 16 pull requests were merged for this release. - gh-13392: BUG: Some PyPy versions lack PyStructSequence_InitType2. - gh-13394: MAINT, DEP: Fix deprecated ``assertEquals()`` - gh-13396: BUG: Fix structured_to_unstructured on single-field types (backport) - gh-13549: BLD: Make CI pass again with pytest 4.5 - gh-13552: TST: Register markers in conftest.py. - gh-13559: BUG: Removes ValueError for empty kwargs in arraymultiter_new - gh-13560: BUG: Add TypeError to accepted exceptions in crackfortran. - gh-13561: BUG: Handle subarrays in descr_to_dtype - gh-13562: BUG: Protect generators from log(0.0) - gh-13563: BUG: Always return views from structured_to_unstructured when... - gh-13564: BUG: Catch stderr when checking compiler version - gh-13565: BUG: longdouble(int) does not work - gh-13587: BUG: distutils/system_info.py fix missing subprocess import (#13523) - gh-13620: BUG,DEP: Fix writeable flag setting for arrays without base - gh-13641: MAINT: Prepare for the 1.16.4 release. - gh-13644: BUG: special case object arrays when printing rel-, abs-error Cheers, Charles Harris [View Less]

2 1

numpy.org/neps not refreshing
by Matti Picus May 27, 2019

May 27, 2019

When we merge a PR, a CI job on circleCI updates both https://github.com/numpy/neps and https://github.com/numpy/devdocs. These are meant to be served as github pages at http://www.numpy.org/neps/ and http://www.numpy.org/devdocs respectively. For some reason the devdocs is updating but the neps site is not, http://www.numpy.org/neps is still showing content from May 22 (a week ago). Could someone who is an admin to the numpy organization try to work out what is going on? It would be … [View More]

2 1

acknowledging sponsorship and institutional partners
by Ralf Gommers May 27, 2019

May 27, 2019

Hi all, In [1] I am adding institutional partners and sponsor logos to the numpy.org website. Sebastian made some good comments there, and I think it would be very helpful if we had some guidelines on how we acknowledge sponsorship. Our governance doc has clear text on what an Institutional Partner is, see [2]. We don't have anything written down about sponsorship though. In my open PR I followed the example of Jupyter (see [3]), which lists Institutional Partners first, followed by Sponsors. … [View More]

3 4

__skip_array_function__ discussion summary
by Sebastian Berg May 25, 2019

May 25, 2019

Hi all, This is an attempt from me to wrap up the discussion a bit so that others can chime in if they want to. NumPy 1.17 will ship with `__array_function__` a way for array like projects (dask, cupy) to override almost all numpy functions [0]. This addition is uncontroversial. NumPy 1.17 will _not_ ship with the `__skip_array_funciton__` following a longer dicussion. For those interested, I tried to give an very short overview over the topic below. The discussion here is around the … [View More]addition of `__skip_array_function__` which would allow code to use: np.ones_like.__skip_array_function__(*args) to reuse the current implementation in numpy (i.e. directly call the current code). This can simplify things drastically for some array- likes, since they do not have to provide an alternative implementation. However, PR-13585 [1] sparked a more detailed discussion, since it was going to add the use of `__skip_array_function__` internally in numpy [2]. The issue is exposure of implementation details. If we do not use it internally, a user may implement their own `np.empty_like` and rely on `np.ones_like` to use `np.empty_like` [3] internally. Thus, `np.ones_like(my_array_like)` can work without `my_array_like` having any special code for `np.ones_like`. The PR exposes the issue that if `np.ones_like` is changed to call `np.empty_like.__skip_array_function__` internally, this will break the users `my_array_like` (it will not call their own `np.empty_like` implementation. We could expect users to fix up such breaking changes, but it exposes how fragile the interaction of user types using `__skip_array_function__` and changes in the specific implementation used by numpy can be in some cases. The second option would be to make sure we use `__skip_array_function__` internally, so that users cannot expect `np.ones_like` to work because they made `np.empty_like` work in the above example (does not increase the "API surface" of NumPy). Plus it increases the issue that the numpy code itself is less readable if we use `__skip_array_function__` internally in many/all places. Those two options further have very different goals in mind for the final usage of the protocol. So that right now the solution is to step back, not include the addition and rather gain experience with the NumPy 1.17 release that includes `__array_function__` but not `__skip_array_function`. I hope this may help those interested who did not follow the full discussion, can't say I feel I am very good at summarizing. For details I encourage you to have a look at the PR discussion and the recent mails to the list. Best, Sebastian [0] http://www.numpy.org/neps/nep-0018-array-function-protocol.html#implementat… [1] https://github.com/numpy/numpy/pull/13585 [2] Mostly for slight optimization. [3] It also uses `np.copyto` which can be overridden as well. [View Less]

4 3