Mailman 3 March 2018 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker July 11, 2023

July 11, 2023

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem Feb. 18, 2023

Feb. 18, 2023

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell July 22, 2021

July 22, 2021

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

10 10

Where to discuss NEPs (was: Re: new NEP: np.AbstractArray and np.asabstractarray)
by Nathaniel Smith May 31, 2018

May 31, 2018

On Thu, Mar 8, 2018 at 7:06 AM, Marten van Kerkwijk <m.h.vankerkwijk(a)gmail.com> wrote: > Hi Nathaniel, > > Overall, hugely in favour! For detailed comments, it would be good to > have a link to a PR; could you put that up? Well, there's a PR here: https://github.com/numpy/numpy/pull/10706 But, this raises a question :-). (One which also came up here: https://github.com/numpy/numpy/pull/10704#issuecomment-371684170) There are sensible two workflows we could use (or at least, two that I can think of): 1. We merge updates to the NEPs as we go, so that whatever's in the repo is the current draft. Anyone can go to the NEP webpage at http://numpy.org/neps (WIP, see #10702) to see the latest version of all NEPs, whether accepted, rejected, or in progress. Discussion happens on the mailing list, and line-by-line feedback can be done by quote-replying and commenting on individual lines. From time to time, the NEP author takes all the accumulated feedback, updates the document, and makes a new post to the list to let people know about the updated version. This is how python-dev handles PEPs. 2. We use Github itself to manage the review. The repo only contains "accepted" NEPs; draft NEPs are represented by open PRs, and rejected NEPs are represented by PRs that were closed-without-merging. Discussion uses Github's commenting/review tools, and happens in the PR itself. This is roughly how Rust handles their RFC process, for example: https://github.com/rust-lang/rfcs Trying to do some hybrid version of these seems like it would be pretty painful, so we should pick one. Given that historically we've tried to use the mailing list for substantive features/planning discussions, and that our NEP process has been much closer to workflow 1 than workflow 2 (e.g., there are already a bunch of old NEPs already in the repo that are effectively rejected/withdrawn), I think we should maybe continue that way, and keep discussions here? So my suggestion is discussion should happen on the list, and NEP updates should be merged promptly, or just self-merged. Sound good? -n -- Nathaniel J. Smith -- https://vorpus.org

7 15

PR to add an initializer kwarg to ufunc.reduce (and similar functions)
by Hameer Abbasi April 9, 2018

April 9, 2018

Hello, everyone. I’ve submitted a PR to add a initializer kwarg to ufunc.reduce. This is useful in a few cases, e.g., it allows one to supply a “default” value for identity-less ufunc reductions, and specify an initial value for reductions such as sum (other than zero.) Please feel free to review or leave feedback, (although I think Eric and Marten have picked it apart pretty well). https://github.com/numpy/numpy/pull/10635 Thanks, Hameer Sent from Astro <https://www.helloastro.com> for Mac

5 21

ANN: numpydoc 0.8.0 release
by Ralf Gommers March 30, 2018

March 30, 2018

Hi all, I'm pleased to announce that a new release of numpydoc is available: - package: https://pypi.python.org/pypi/numpydoc - documentation (new in this release): https://numpydoc.readthedocs.io/en/latest/ This is a maintenance release with many small improvements. Likely your documentation will render with fewer warnings; e.g. for NumPy it removed ~300 irrelevant ones, while improving the rendered results. Thanks to everyone who contributed to this release! Cheers, Ralf

1 0

NEUROLAB - SPYDER
by Fernando Fernandes Vieira March 30, 2018

March 30, 2018

Hello everyone How to install neurolab in spyder? Can someone help me Att.. _______________________________________________________ * FERNANDO FERNANDES VIEIRA* Departamento de Engenharia Sanitária e Ambiental - DESA Centro de Ciências e Tecnologia - CCT Universidade Estadual da Paraíba - UEPB Tel: (83) 3315-3333 (DESA) - (83) 98852-1461 (Pessoal) e-mail: fernando(a)uepb.edu.br (fernanvieira(a)gmail.com) Campina Grande - PB - Brasil _______________________________________________________

2 2

best way of speeding up a filtering-like algorithm
by Moroney, Catherine M (398E) March 29, 2018

March 29, 2018

Hello, I have the following sample code (pretty simple algorithm that uses a rolling filter window) and am wondering what the best way is of speeding it up. I tried rewriting it in Cython by pre-declaring the variables but that didn’t buy me a lot of time. Then I rewrote it in Fortran (and compiled it with f2py) and now it’s lightning fast. But I would still like to know if I could rewrite it in pure python/numpy/scipy or in Cython and get a similar speedup. Here is the raw Python code: def mixed_coastline_slow(nsidc, radius, count, mask=None): nsidc_copy = numpy.copy(nsidc) if (mask is None): idx_coastline = numpy.where(nsidc_copy == NSIDC_COASTLINE_MIXED) else: idx_coastline = numpy.where(mask & (nsidc_copy == NSIDC_COASTLINE_MIXED)) for (irow0, icol0) in zip(idx_coastline[0], idx_coastline[1]): rows = ( max(irow0-radius, 0), min(irow0+radius+1, nsidc_copy.shape[0]) ) cols = ( max(icol0-radius, 0), min(icol0+radius+1, nsidc_copy.shape[1]) ) window = nsidc[rows[0]:rows[1], cols[0]:cols[1]] npoints = numpy.where(window != NSIDC_COASTLINE_MIXED, True, False).sum() nsnowice = numpy.where( (window >= NSIDC_SEAICE_LOW) & (window <= NSIDC_FRESHSNOW), \ True, False).sum() if (100.0*nsnowice/npoints >= count): nsidc_copy[irow0, icol0] = MISR_SEAICE_THRESHOLD return nsidc_copy and here is my attempt at Cython-izing it: import numpy cimport numpy as cnumpy cimport cython cdef int NSIDC_SIZE = 721 cdef int NSIDC_NO_SNOW = 0 cdef int NSIDC_ALL_SNOW = 100 cdef int NSIDC_FRESHSNOW = 103 cdef int NSIDC_PERMSNOW = 101 cdef int NSIDC_SEAICE_LOW = 1 cdef int NSIDC_SEAICE_HIGH = 100 cdef int NSIDC_COASTLINE_MIXED = 252 cdef int NSIDC_SUSPECT_ICE = 253 cdef int MISR_SEAICE_THRESHOLD = 6 def mixed_coastline(cnumpy.ndarray[cnumpy.uint8_t, ndim=2] nsidc, int radius, int count): cdef int irow, icol, irow1, irow2, icol1, icol2, npoints, nsnowice cdef cnumpy.ndarray[cnumpy.uint8_t, ndim=2] nsidc2 \ = numpy.empty(shape=(NSIDC_SIZE, NSIDC_SIZE), dtype=numpy.uint8) cdef cnumpy.ndarray[cnumpy.uint8_t, ndim=2] window \ = numpy.empty(shape=(2*radius+1, 2*radius+1), dtype=numpy.uint8) nsidc2 = numpy.copy(nsidc) idx_coastline = numpy.where(nsidc2 == NSIDC_COASTLINE_MIXED) for (irow, icol) in zip(idx_coastline[0], idx_coastline[1]): irow1 = max(irow-radius, 0) irow2 = min(irow+radius+1, NSIDC_SIZE) icol1 = max(icol-radius, 0) icol2 = min(icol+radius+1, NSIDC_SIZE) window = nsidc[irow1:irow2, icol1:icol2] npoints = numpy.where(window != NSIDC_COASTLINE_MIXED, True, False).sum() nsnowice = numpy.where( (window >= NSIDC_SEAICE_LOW) & (window <= NSIDC_FRESHSNOW), \ True, False).sum() if (100.0*nsnowice/npoints >= count): nsidc2[irow, icol] = MISR_SEAICE_THRESHOLD return nsidc2 Thanks in advance for any advice! Catherine

6 6

PR adding support for object arrays to np.isinf, np.isnan, np.isfinite
by Joseph Fox-Rabinovitz March 28, 2018

March 28, 2018

I have opened PR #10820 to add support for `dtype=object` to `np.isinf`, `np.isnan`, `np.isfinite`. The PR is a fairly minor change, but I would like to make sure that I understand at least the basics of ufuncs before I start adding support for datetimes and timedeltas to `np.isfinite` and eventually to `np.histogram`. I have left a few comments in areas I am not sure about, and would greatly appreciate feedback, even if the PR is not found suitable for merging. With this PR, object arrays containing any numerical or simulated numerical types (implementing `__float__` or `__complex__` methods) are processed as would be expected. While working on PR, I came up with two questions for the gurus: 1. Am I correct in understanding that `isinf`, `isnan` and `isfinite` currently cast integer inputs to float to process them? Why are integer inputs not optimized to return arrays of all False, False, True, respectively for those functions? 2. Why are `isneginf` and `isposinf` not ufuncs? Is there any reason not to make them ufuncs (besides the renaming of the `y` parameter to `out`, which technically breaks some backward compatibility)? Regards, - Joe

1 0

round(numpy.float64(0.0)) is a numpy.float64
by Olivier March 26, 2018

March 26, 2018

Hello, Is it normal, expected and desired that : round(numpy.float64(0.0)) is a numpy.float64 while round(numpy.float(0.0)) is an integer? I find it disturbing and misleading. What do you think? Has it already been discussed somewhere else? Best regards, Olivier

5 6