Mailman 3 July 2016 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker July 11, 2023

July 11, 2023

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem Feb. 18, 2023

Feb. 18, 2023

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How … [View More]

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell July 22, 2021

July 22, 2021

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program … [View More]makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil [View Less]

10 10

Added atleast_nd, request for clarification/cleanup of atleast_3d
by Joseph Fox-Rabinovitz Oct. 27, 2016

Oct. 27, 2016

Hi, I have generalized np.atleast_1d, np.atleast_2d, np.atleast_3d with a function np.atleast_nd in PR#7804 (https://github.com/numpy/numpy/pull/7804). As a result of this PR, I have a couple of questions about `np.atleast_3d`. `np.atleast_3d` appears to do something weird with the dimensions: If the input is 1D, it prepends and appends a size-1 dimension. If the input is 2D, it appends a size-1 dimension. This is inconsistent with `np.atleast_2d`, which always prepends (as does `np.… [View More]

11 25

ufunc reduceat behavior on empty slices
by Erik Brinkman July 29, 2016

July 29, 2016

Hi, The behavior of a ufuncs reduceat on empty slices seems a little strange, and I wonder if there's a reason behind it / if there's a route to potentially changing it. First, I'll go into a little background. I've been making a lot of use of ufuncs reduceat functionality on staggered arrays. In general, I'll have "n" arrays, each with size "s[n]" and I'll store them in one array "x", such that "s.sum() == x.size". reduceat is great because I use ufunc.reduceat(x, np.insert(s[:-1].cumsum(), … [View More]0, 0)) to get some summary information about each array. However, reduceat seems to behave strangely for empty slices. To make things concrete, let's assume: import numpy as np s = np.array([3, 0, 2]) x = np.arange(s.sum()) inds = np.insert(s[:-1].cumsum(), 0, 0) # [0, 3, 3] np.add.reduceat(x, inds) # [3, 3, 7] not [3, 0, 7] # This is distinct from np.fromiter(map(np.add.reduce, np.array_split(x, inds[1:])), x.dtype, s.size - 1) # [3, 0, 7] what I wanted The current documentation <http://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.reduceat.ht…> on reduceat first states: For i in range(len(indices)), reduceat computes ufunc.reduce(a[indices[i]:indices[i+1]]) That would suggest the outcome, that I expected. However, in the examples section it goes into a bunch of examples which contradict that statement and instead suggest that the actual algorithm is more akin to: ufunc.reduce(a[indices[i]:indices[i+1]]) if indices[i+1] > indices[i] else indices[i] Looking at the source <https://github.com/numpy/numpy/blob/2af06c804931aae4b30bb3349bc60271b0b6538…>, it seems like it's copying indices[i], and then while there are more elements to process it keeps reducing, resulting in this unexpected behavior. It seems like the proper thing to do would be start with ufunc.identity, and then reduce. This is slightly less performant than what's implemented, but more "correct." There could, of course, just be a switch to copy the identity only when the slice is empty. Is there a reason it's implemented like this? Is it just for performance, or is this strange behavior *useful* somewhere? It seems like "fixing" this would be bad because you'll be changing somewhat documented functionality in a backwards incompatible way. What would the best approach to "fixing" this be? add another function "reduceat_"? add a flag to reduceat to do the proper thing for empty slices? Finally, is there a good way to work around this? I think for now I'm just going to mask out the empty slices and use insert to add them back in, but if I'm missing an obvious solution, I'll look at that too. I need to mask them out because, np.add.reduceat(x, 5) would ideally return 0, but instead it throws an error since 5 is out of range... Thanks for indulging my curiosity, Erik [View Less]

2 1

Is there any official position on PEP484/mypy?
by Daniel Moisset July 29, 2016

July 29, 2016

Hi, I work at Machinalis were we use a lot of numpy (and the pydata stack in general). Recently we've also been getting involved with mypy, which is a tool to type check (not on runtime, think of it as a linter) annotated python code (the way of annotating python types has been recently standarized in PEP 484). As part of that involvement we've started creating type annotations for the Python libraries we use most, which include numpy. Mypy provides a way to specify types with annotations in … [View More]

4 5

ANN: Bokeh 0.12.1 released
by Bryan Van de Ven July 28, 2016

July 28, 2016

Hi all, On behalf of the Bokeh team, I am pleased to announce the release of version 0.12.1 of Bokeh! This is a minor, incremental update that adds a few new small features and fixes several bugs. Please see the announcement post at: https://bokeh.github.io/blog/2016/6/28/release-0-12-1/ which has much more information as well as live demonstrations. And as always, see the CHANGELOG and Release Notes for full details. If you are using Anaconda/miniconda, you can install it with conda: … [View More]

1 0

f2py NotImplementedError
by andy buzza July 26, 2016

July 26, 2016

Hello all, I have been trying to compile a relatively simple pair of Fortran files, one referencing a subroutine from another file (mainmodule.f90 references othermodule.f90). I have been able to compile them using a Fortran compiler, but receive a NotImplementedError when using f2py. Steps I use for f2py: $gfortran -shared -o othermodule.so othermodule.f90 -fPIC $f2py -c -l/path/othermodule -m mainmodule mainmodule.f90 I am running this on linux and wasn't sure how to correct the error. … [View More]

2 1

ANN: scipy 0.18.0 release
by Evgeni Burovski July 25, 2016

July 25, 2016

On behalf of the Scipy development team I am pleased to announce the availability of Scipy 0.18.0. This release contains several great new features and a large number of bug fixes and various improvements, as detailed in the release notes below. 99 people contributed to this release over the course of six months. Thanks to everyone who contributed! This release requires Python 2.7 or 3.4-3.5 and NumPy 1.7.2 or greater. Source tarballs and release notes can be found at https://github.com/scipy/… [View More]

1 0

Re: [Numpy-discussion] Performance issue in covariance function in Numpy 1.9 and later
by Ecem sogancıoglu July 22, 2016

July 22, 2016

Hi Ralf, Thank you so much for your answer. I finally figured out that the problem was because Numpy 1.9 was not linked to BLAS. I do not know why because I simply installed numpy 1.9 via the commands: apt-get install python3-numpy If anybody has the same problem, you may want to take a look into this: http://osdf.github.io/blog/numpyscipy-with-openblas-for-ubuntu-1204-second-… Best Regards, Ecem On Fri, Jul 22, 2016 at 5:19 PM Ecem sogancıoglu <ecemsogancioglu(a)gmail.com> wrote: … [View More]

1 0