Mailman 3 October 2015 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker 11 Jul '23

11 Jul '23

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem 18 Feb '23

18 Feb '23

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell 22 Jul '21

22 Jul '21

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

10 10

C-API: multidimensional array indexing?
by Johann Bauer 31 Mar '16

31 Mar '16

Dear experts, is there a C-API function for numpy which implements Python's multidimensional indexing? Say, I have a 2d-array PyArrayObject * M; and an index int i; how do I extract the i-th row or column M[i,:] respectively M[:,i]? I am looking for a function which gives again a PyArrayObject * and which is a view to M (no copied data; the result should be another PyArrayObject whose data and strides points to the correct memory portion of M). I searched the API documentation, Google and mailing lists for quite a long time but didn't find anything. Can you help me? Thanks, Johann

3 3

Generalize hstack/vstack --> stack; Block matrices like in matlab
by Stefan Otte 09 Jan '16

09 Jan '16

Hey, quite often I work with block matrices. Matlab offers the convenient notation [ a b; c d ] to stack matrices. The numpy equivalent is kinda clumsy: vstack([hstack([a,b]), hstack([c,d])]) I wrote the little function `stack` that does exactly that: stack([[a, b], [c, d]]) In my case `stack` replaced `hstack` and `vstack` almost completely. If you're interested in including it in numpy I created a pull request [1]. I'm looking forward to getting some feedback! Best, Stefan [1] https://github.com/numpy/numpy/pull/5057

12 24

numpy-1.11.0.dev0 windows wheels compiled with mingwpy available
by Carl Kleffner 23 Dec '15

23 Dec '15

I made numpy master (numpy-1.11.0.dev0 , https://github.com/numpy/numpy/commit/0243bce23383ff5e894b99e40df2f8fd806ad…) windows binary wheels available for testing. Install it with pip: > pip install -i https://pypi.anaconda.org/carlkl/simple numpy These builds are compiled with OPENBLAS trunk for BLAS/LAPACK support and the mingwpy compiler toolchain. OpenBLAS is deployed within the numpy wheels. To be performant on all usual CPU architectures OpenBLAS is configured with it's 'dynamic architecture' and automatic CPU detection. This version of numpy fakes long double as double just like the MSVC builds. Some test statistics: win32 (32 bit) numpy-1.11.0.dev0, python-2.6: errors=8, failures=1 numpy-1.11.0.dev0, python-2.7: errors=8, failures=1 numpy-1.11.0.dev0, python-3.3: errors=9 numpy-1.11.0.dev0, python-3.4: errors=9 amd64 (64bit) numpy-1.11.0.dev0, python-2.6: errors=9, failures=6 numpy-1.11.0.dev0, python-2.7: errors=9, failures=6 numpy-1.11.0.dev0, python-3.3: errors=10, failures=6 numpy-1.11.0.dev0, python-3.4: errors=10, failures=6 Carl

6 7

future of f2py and Fortran90+
by Eric Firing 04 Dec '15

04 Dec '15

F2py is a great tool, but my impression is that it is being left behind by the evolution of Fortran from F90 onward. This is unfortunate; it would be nice to be able to easily wrap new Fortran libraries. I'm curious: has anyone been looking into what it would take to enable f2py to handle modern Fortran in general? And into prospects for getting such an effort funded? Eric

5 6

Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead
by Nathaniel Smith 06 Nov '15

06 Nov '15

Hi all, Apparently it is not well known that if you have a Python project source tree (e.g., a numpy checkout), then the correct way to install it is NOT to type python setup.py install # bad and broken! but rather to type pip install . (I.e., pip install isn't just for packages on pypi -- you can also pass it the path to an arbitrary source directory or the URL of a source tarball and it will do its thing. In this case "install ." means "install the project in the current directory".) These don't quite have identical results -- the main difference is that the latter makes sure that proper metadata gets installed so that later on it will be possible to upgrade or uninstall correctly. If you call setup.py directly, and then later you try to upgrade your package, then it's entirely possible to end up with a mixture of old and new versions of the package installed in your PYTHONPATH. (One common effect is in numpy's case is that we get people sending us mysterious bug reports about failing tests in files don't even exist (!) -- because nose is finding tests in files from one version of numpy and running them against a different version of numpy.) But this isn't the only issue -- using pip also avoids a bunch of weird corner cases in distutils/setuptools. E.g., if setup.py uses plain distutils, then it turns out this will mangle numpy version numbers in ways that cause weird horribleness -- see [1] for a bug report of the form "matplotlib doesn't build anymore" which turned out to be because of using 'setup.py install' to install numpy. OTOH if setup.py uses setuptools then you get different weirdnesses, like you can easily end up with multiple versions of the same library installed simultaneously. And finally, an advantage of getting used to using 'pip install .' now is that you'll be prepared for the glorious future when we kill distutils and get rid of setup.py entirely in favor of something less terrible [2]. So a proposal that came out of the discussion in [1] is that we modify numpy's setup.py now so that if you try running python setup.py install you get Error: Calling 'setup.py install' directly is NOT SUPPORTED! Instead, do: pip install . Alternatively, if you want to proceed at your own risk, you can try 'setup.py install --force-raw-setup.py' For more information see http://... (Other setup.py commands would continue to work as normal.) I believe that this would also break both 'easy_install numpy', and attempts to install numpy via the setup_requires= argument to setuptools.setup (because setup_requires= implicitly calls easy_install). install_requires= would *not* be affected, and setup_requires= would still be fine in cases where numpy was already installed. This would hopefully cut down on the amount of time everyone spends trying to track down these stupid weird bugs, but it will also require some adjustment in people's workflows, so... objections? concerns? -n [1] https://github.com/numpy/numpy/issues/6551 [2] https://mail.python.org/pipermail/distutils-sig/2015-October/027360.html -- Nathaniel J. Smith -- http://vorpus.org

15 40

deprecate fromstring() for text reading?
by Chris Barker 04 Nov '15

04 Nov '15

There was just a question about a bug/issue with scipy.fromstring (which is numpy.fromstring) when used to read integers from a text file. https://mail.scipy.org/pipermail/scipy-user/2015-October/036746.html fromstring() is bugging and inflexible for reading text files -- and it is a very, very ugly mess of code. I dug into it a while back, and gave up -- just to much of a mess! So we really should completely re-implement it, or deprecate it. I doubt anyone is going to do a big refactor, so that means deprecating it. Also -- if we do want a fast read numbers from text files function (which would be nice, actually), it really should get a new name anyway. (and the hopefully coming new dtype system would make it easier to write cleanly) I'm not sure what deprecating something means, though -- have it raise a deprecation warning in the next version? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

8 12

[NumPy/Swig] Return NumPy array with same size as input array (no additional length argument)
by laurentes 02 Nov '15

02 Nov '15

Hello, Using Swig, I don't manage to (properly) create the Python Binding for the following C-like function: void add_array(double* input_array1, double* input_array2, double* output_array, int length); where the three arrays have all the same length. This is similar to this thread <http://numpy-discussion.10968.n7.nabble.com/Numpy-SWIG-td25709.html#a25710> , which has never been fully addressed online. >From Python, I would like to be able to call: add_array(input_array1, input_array2) which would return me a newly allocated NumPy array (output_array) with the result. In my Swig file, I've first used the wrapper function trick described here <http://web.mit.edu/6.863/spring2011/packages/numpy_src/doc/swig/doc/numpy_s…> , that is: %apply (double* IN_ARRAY1, int DIM1) {(double* input_array1, int length1), (double* input_array2, int length2)}; %apply (double* ARGOUT_ARRAY1, int DIM1) {(double* output_array, int length3)}; %rename (add_array) my_add_array; %exception my_add_array { $action if (PyErr_Occurred()) SWIG_fail; } %inline %{ void my_add_array(double* input_array1, int length1, double* input_array2, int length2, double* output_array, int length3) { if (length1 != length2 || length1 != length3) { PyErr_Format(PyExc_ValueError, "Arrays of lengths (%d,%d,%d) given", length1, length2, length3); } else { add_array(input_array1, input_array2, output_array, length1); } } %} This allows me to call the function from Python using add_array(input_array1, input_array2, length). But the third argument of this function is useless and this function does not look 'Pythonic'. Could someone help me to modify my Swig file, such that only the first two arguments are required for the Python API? Thanks a lot, Laurent -- View this message in context: http://numpy-discussion.10968.n7.nabble.com/NumPy-Swig-Return-NumPy-array-w… Sent from the Numpy-discussion mailing list archive at Nabble.com.

2 1