Mailman 3 August 2017 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker 11 Jul '23

11 Jul '23

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem 18 Feb '23

18 Feb '23

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell 22 Jul '21

22 Jul '21

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

10 10

Dropping support for Accelerate
by Ilhan Polat 16 Sep '17

16 Sep '17

A few months ago, I had the innocent intention to wrap LDLt decomposition routines of LAPACK into SciPy but then I am made aware that the minimum required version of LAPACK/BLAS was due to Accelerate framework. Since then I've been following the core SciPy team and others' discussion on this issue. We have been exchanging opinions for quite a while now within various SciPy issues and PRs about the ever-increasing Accelerate-related issues and I've compiled a brief summary about the ongoing discussions to reduce the clutter. First, I would like to kindly invite everyone to contribute and sharpen the cases presented here https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate The reason I specifically wanted to post this also in NumPy mailing list is to probe for the situation from the NumPy-Accelerate perspective. Is there any NumPy specific problem that would indirectly effect SciPy should the support for Accelerate is dropped?

5 16

np.array, copy=False and memmap
by Nisoli Isaia 06 Sep '17

06 Sep '17

Dear all, I have a question about the behaviour of y = np.array(x, copy=False, dtype='float32') when x is a memmap. If we check the memmap attribute of mmap print "mmap attribute", y._mmap numpy tells us that y is not a memmap. But the following code snippet crashes the python interpreter # opens the memmap with open(filename,'r+b') as f: mm = mmap.mmap(f.fileno(),0) x = np.frombuffer(mm, dtype='float32') # builds an array from the memmap, with the option copy=False y = np.array(x, copy=False, dtype='float32') print "before", y # closes the file mm.close() print "after", y In my code I use memmaps to share read-only objects when doing parallel processing and the behaviour of np.array, even if not consistent, it's desirable. I share scipy sparse matrices over many processes and if np.array would make a copy when dealing with memmaps this would force me to rewrite part of the sparse matrices code. Would it be possible in the future releases of numpy to have np.array check, if copy is false, if y is a memmap and in that case return a full memmap object instead of slicing it? Best wishes Isaia P.S. A longer account of the issue may be found on my university blog http://www.im.ufrj.br/nisoli/blog/?p=131 -- Isaia Nisoli

4 5

selecting all 2-d slices out of n-dimensional array
by Moroney, Catherine M (398E) 30 Aug '17

30 Aug '17

Hello, I have an n-dimensional array (say (4,4,2,2)) and I wish to automatically extract all the (4,4) slices in it. i.e. a = numpy.arange(0, 64).reshape(4,4,2,2) slice1 = a[..., 0, 0] slice2 = a[..., 0, 1] slice3 = a[..., 1, 0] slice4 = a[..., 1,1] Simple enough example but in my case array “a” will have unknown rank and size. All I know is that it will have more than 2 dimensions, but I don’t know ahead of time how many dimensions or what the size of those dimensions are. What is the best way of tackling this problem without writing a whole bunch of if-then cases depending on what the rank and shape of a is? Is there a one-size-fits-all solution? I’m using python 2.7 and numpy 1.8.2 Thanks for any advice, Catherine

3 2

Interface numpy arrays to Matlab?
by Neal Becker 29 Aug '17

29 Aug '17

I've searched but haven't found any decent answer. I need to call Matlab from python. Matlab has a python module for this purpose, but it doesn't understand numpy AFAICT. What solutions are there for efficiently interfacing numpy arrays to Matlab? Thanks, Neal

6 6

power function distribution or power-law distribution?
by Renato Fabbri 24 Aug '17

24 Aug '17

numpy.random.power.__doc__ uses only the term "power function distribution". I cannot find a comparison between this term and "power-law distribution" and am quite interested to know if they are simply synonyms. Any ideas? BTW. how is this list related to numpy-discussion(a)scipy.org? -- Renato Fabbri GNU/Linux User #479299 labmacambira.sourceforge.net

5 7

Why are empty arrays False?
by Michael Lamparski 22 Aug '17

22 Aug '17

Greetings, all. I am troubled. The TL;DR is that `bool(array([])) is False` is misleading, dangerous, and unnecessary. Let's begin with some examples: >>> bool(np.array(1)) True >>> bool(np.array(0)) False >>> bool(np.array([0, 1])) ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() >>> bool(np.array([1])) True >>> bool(np.array([0])) False >>> bool(np.array([])) False One of these things is not like the other. The first three results embody a design that is consistent with some of the most fundamental design choices in numpy, such as the choice to have comparison operators like `==` work elementwise. And it is the only such design I can think of that is consistent in all edge cases. (see footnote 1) The next two examples (involving arrays of shape (1,)) are a straightforward extension of the design to arrays that are isomorphic to scalars. I can't say I recall ever finding a use for this feature... but it seems fairly harmless. So how about that last example, with array([])? Well... it's /kind of/ like how other python containers work, right? Falseness is emptiness (see footnote 2)... Except that this is actually *a complete lie*, due to /all of the other examples above/! Here's what I would like to see: >>> bool(np.array([])) ValueError: The truth value of a non-scalar array is ambiguous. Use a.any() or a.all() Why do I care? Well, I myself wasted an hour barking up the wrong tree while debugging some code when it turned out that I was mistakenly using truthiness to identify empty arrays. It just so happened that the arrays always contained 1 or 0 elements, so it /appeared/ to work except in the rare case of array([0]) where things suddenly exploded. I posit that there is no usage of the fact that `bool(array([])) is False` in any real-world code which is not accompanied by a horrible bug writhing in hiding just beneath the surface. For this reason, I wish to see this behavior *abolished*. Thank you. -Michael Footnotes: 1: Every now and then, I wish that `ndarray.__{bool,nonzero}__` would just implicitly do `all()`, which would make `if a == b:` work like it does for virtually every other reasonably-designed type in existence. But then I recall that, if this were done, then the behavior of `if a != b:` would stand out like a sore thumb instead. Truly, punting on 'any/all' was the right choice. 2: np.array([[[[]]]]) is also False, which makes this an interesting sort of n-dimensional emptiness test; but if that's really what you're looking for, you can achieve this much more safely with `np.all(x.shape)` or `bool(x.flat)`

10 19

Re: [Numpy-discussion] Why are empty arrays False?
by Andrew Nelson 19 Aug '17

19 Aug '17

> I think the consistency between bool([]) and bool(array([])) is worth preserving I'm with Eric Firing on this one. Empty sequences are False in Python.

2 1