Mailman 3 December 2018 - NumPy-Discussion

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem 18 Feb '23

18 Feb '23

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell 22 Jul '21

22 Jul '21

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

10 10

Add guaranteed no-copy to array creation and reshape?
by Sebastian Berg 13 Jan '19

13 Jan '19

Hi all, In https://github.com/numpy/numpy/pull/11897 I am looking into the addition of a `copy=np.never_copy` argument to: * np.array * arr.reshape/np.reshape * arr.astype Which would cause an error to be raised when numpy cannot guarantee that the returned array is a view of the input array. The motivation is to easier avoid accidental copies of large data, or ensure that in-place manipulation will be meaningful. The copy flag API would be: * `copy=True` forces a copy * `copy=False` allows numpy to copy if necessary * `copy=np.never_copy` will error if a copy would be necessary * (almost) all other input will be deprecated. Unfortunately using `copy="never"` is tricky, because currently `np.array(..., copy="never")` behaves exactly the same as `np.array(..., copy=bool("never"))`. So that the wrong result would be given on old numpy versions and it would be a behaviour change. Some things that are a not so nice maybe: * adding/using `np.never_copy` is not very nice * Scalars need a copy and so will not be allowed * For rare array-likes numpy may not be able to guarantee no-copy, although it could happen (but should not). The history is that a long while ago I considered adding a copy flag to `reshape` so that it is possible to do `copy=np.never_copy` (or similar) to ensure that no copy is made. In these, you may want something like an assertion: ``` new_arr = arr.reshape(new_shape) assert np.may_share_memory(arr, new_arr) # Which is sometimes -- but should not be -- written as: arr.shape = new_shape # unnecessary container modification # Or: view = np.array(arr, order="F") assert np.may_share_memory(arr, new_arr) ``` but is more readable and will not cause an intermediate copy on error. So what do you think? Other variants would be to not expose this for `np.array` and probably limit `copy="never"` to the reshape method. Or just to not do it at all. Or to also accept "never" for `reshape`, although I think I would prefer to keep it in sync and wait for a few years to consider that. Best, Sebastian

9 32

three-dim array
by Jeff 27 Dec '18

27 Dec '18

hello, sorry newbe to numpy. I want to define a three-dim array. I know this works: >>> np.array([[[1,2],[3,4]],[[5,6],[7,8]]]) array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]]) But can you tell why this doesnt work? >>> np.array([[1,2],[[1,2],[3,4]]]) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: setting an array element with a sequence. Thank you.

6 6

ANN: SfePy 2018.4
by Robert Cimrman 26 Dec '18

26 Dec '18

I am pleased to announce release 2018.4 of SfePy. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (limited support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: https://mail.python.org/mm3/mailman3/lists/sfepy.python.org/ Git (source) repository, issue tracker: https://github.com/sfepy/sfepy Highlights of this release -------------------------- - better support for eigenvalue problems - improved MUMPS solver interface - support for logging and plotting of complex values For full release notes see [1]. Cheers, Robert Cimrman [1] http://docs.sfepy.org/doc/release_notes.html#id1 --- Contributors to this release in alphabetical order: Robert Cimrman Vladimir Lukes Matyas Novak Jan Heczko Lubos Kejzlar

1 0

Warn or immidiately change readonly flag on broadcast_arrays return value?
by Matti Picus 25 Dec '18

25 Dec '18

In PR 12609 https://github.com/numpy/numpy/pull/12609 I added code to emit a DepricationWarning when broadcast_arrays returns an array where the output is repeated. While this is a minimal fix to the problem, perhaps we should consider making the output readonly immediately instead? - A deprecation cycle requires two changes to downstream user's code: one to filter the deprecation warning, and another when we actually make the change - Writing to the repeated data will cause errors now. What do you think, should we change the behaviour at all, and if so should we depricate it over two releases or change it immediately? The original issue is here https://github.com/numpy/numpy/issues/2705 Matti

4 3

ANN: NumExpr 2.6.9
by Robert McLeod 21 Dec '18

21 Dec '18

Hi everyone, This is a version-bump release to provide wheels for Python 3.7.1 on Windows platforms. Also Mike Toews made the handling of our environment variables more robust. Project documentation is available at: http://numexpr.readthedocs.io/ Changes from 2.6.8 to 2.6.9 --------------------------- - Thanks to Mike Toews for more robust handling of the thread-setting environment variables. - With Appveyor updating to Python 3.7.1, wheels for Python 3.7 are now available in addition to those for other OSes. What's Numexpr? --------------- Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It has multi-threaded capabilities, as well as support for Intel's MKL (Math Kernel Library), which allows an extremely fast evaluation of transcendental functions (sin, cos, tan, exp, log...) while squeezing the last drop of performance out of your multi-core processors. Look here for a some benchmarks of numexpr using MKL: https://github.com/pydata/numexpr/wiki/NumexprMKL Its only dependency is NumPy (MKL is optional), so it works well as an easy-to-deploy, easy-to-use, computational engine for projects that don't want to adopt other solutions requiring more heavy dependencies. Where I can find Numexpr? ------------------------- The project is hosted at GitHub in: https://github.com/pydata/numexpr You can get the packages from PyPI as well (but not for RC releases): http://pypi.python.org/pypi/numexpr Documentation is hosted at: http://numexpr.readthedocs.io/en/latest/ Share your experience --------------------- Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data! -- Robert McLeod, Ph.D. robbmcleod(a)gmail.com robbmcleod(a)protonmail.com robert.mcleod(a)hitachi-hhtc.ca www.entropyreduction.al

1 0

NumPy 1.16.0rc1 released
by Charles R Harris 20 Dec '18

20 Dec '18

Hi All, On behalf of the NumPy team I'm pleased to announce the release of NumPy 1.16.0rc1. This is the last NumPy release to support Python 2.7 and will be maintained as a long term release with bug fixes until 2020. This release has seen a lot of refactoring and features many bug fixes, improved code organization, and better cross platform compatibility. Not all of these improvements will be visible to users, but they should help make maintenance easier going forward. Highlights are - Experimental support for overriding numpy functions in downstream projects. - The matmul function is now a ufunc and can be overridden using __array_ufunc__. - Improved support for the ARM and POWER architectures. - Improved support for AIX and PyPy. - Improved interoperation with ctypes. - Improved support for PEP 3118. The supported Python versions are 2.7 and 3.5-3.7, support for 3.4 has been dropped. The wheels on PyPI are linked with OpenBLAS v0.3.4+, which should fix the known threading issues found in previous OpenBLAS versions. Downstream developers building this release should use Cython >= 0.29 and, if linking OpenBLAS, OpenBLAS > v0.3.4. Wheels for this release can be downloaded from PyPI <https://pypi.org/project/numpy/1.16.0rc1/>, source archives are available from Github <https://github.com/numpy/numpy/releases/tag/v1.16.0rc1>. *Contributors* A total of 111 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Alan Fontenot + - Allan Haldane - Alon Hershenhorn + - Alyssa Quek + - Andreas Nussbaumer + - Anner + - Anthony Sottile + - Antony Lee - Ayappan P + - Bas van Schaik + - C.A.M. Gerlach + - Charles Harris - Chris Billington - Christian Clauss - Christoph Gohlke - Christopher Pezley + - Daniel B Allan + - Daniel Smith - Dawid Zych + - Derek Kim + - Dima Pasechnik + - Edgar Giovanni Lepe + - Elena Mokeeva + - Elliott Sales de Andrade + - Emil Hessman + - Eric Schles + - Eric Wieser - Giulio Benetti + - Guillaume Gautier + - Guo Ci - Heath Henley + - Isuru Fernando + - J. Lewis Muir + - Jack Vreeken + - Jaime Fernandez - James Bourbeau - Jeff VanOss - Jeffrey Yancey + - Jeremy Chen + - Jeremy Manning + - Jeroen Demeyer - John Darbyshire + - John Zwinck - Jonas Jensen + - Joscha Reimer + - Juan Azcarreta + - Julian Taylor - Kevin Sheppard - Krzysztof Chomski + - Kyle Sunden - Lars Grüter - Lilian Besson + - MSeifert04 - Mark Harfouche - Marten van Kerkwijk - Martin Thoma - Matt Harrigan + - Matthew Bowden + - Matthew Brett - Matthias Bussonnier - Matti Picus - Max Aifer + - Michael Hirsch, Ph.D + - Michael James Jamie Schnaitter + - MichaelSaah + - Mike Toews - Minkyu Lee + - Mircea Akos Bruma + - Mircea-Akos Brumă + - Moshe Looks + - Muhammad Kasim + - Nathaniel J. Smith - Nikita Titov + - Paul Müller + - Paul van Mulbregt - Pauli Virtanen - Pierre Glaser + - Pim de Haan - Ralf Gommers - Robert Kern - Robin Aggleton + - Rohit Pandey + - Roman Yurchak + - Ryan Soklaski - Sebastian Berg - Sho Nakamura + - Simon Gibbons - Stan Seibert + - Stefan Otte - Stefan van der Walt - Stephan Hoyer - Stuart Archibald - Taylor Smith + - Tim Felgentreff + - Tim Swast + - Tim Teichmann + - Toshiki Kataoka - Travis Oliphant - Tyler Reddy - Uddeshya Singh + - Warren Weckesser - Weitang Li + - Wenjamin Petrenko + - William D. Irons - Yannick Jadoul + - Yaroslav Halchenko - Yug Khanna + - Yuji Kanagawa + - Yukun Guo + - lerbuke + - @ankokumoyashi + Cheers, Charles Harris

1 0

Adding "maximum difference" to np.testing.assert_array_equal errors
by Stephan Hoyer 20 Dec '18

20 Dec '18

I have a PR up for review that adds "maximum difference" to the error messages produced by NumPy's testing functions for comparing arrays: https://github.com/numpy/numpy/pull/12591 Because this changes NumPy's public interface, I'm running it by the mailing list to see if there are any objections or suggestions for improvement. Example behavior: >>> x = np.array([1, 2, 3]) >>> y = np.array([1, 2, 3.0001]) >>> np.testing.assert_allclose(x, y) AssertionError: Not equal to tolerance rtol=1e-07, atol=0 (mismatch 33.333333333333336%, maximum difference 0.00010000000000021103) x: array([1, 2, 3]) y: array([1. , 2. , 3.0001]) Motivation: when writing numerical algorithms, I frequently find myself experimenting to pick the right value of atol and rtol for np.testing.assert_allclose(). If I make the tolerance too generous, I risk missing regressions in accuracy, so I usually try to pick the smallest values for which tests pass. This change immediately reveals appropriate values to use for these parameters, so I don't need to guess and check. One alternative would be print both "atol" and "rtol" numbers directly that could be immediately used in assert_allclose(). Here we effectively only have "atol". The main reason why I didn't do this is that I felt it would add more clutter (rtol is slightly redundant with the displayed array values). But we could probably do this if we're willing to split the error message onto a few more lines, e.g., AssertionError: Not equal to tolerance rtol=1e-07, atol=0 (mismatch 33.333333333333336%, maximum absolute difference 0.00010000000000021103, maximum relative difference 3.333333333340368e-05) x: array([1, 2, 3]) y: array([1. , 2. , 3.0001]) Best, Stephan

4 4

Reminder: weekly status meeting Dec 19 at 12:00 pacific time
by Matti Picus 19 Dec '18

19 Dec '18

The draft agenda is at https://hackmd.io/Gn1ymjwkRjm9WVY5Cgbwsw?both This will be our last meeting of 2018. The BIDS team

1 0

NumPy-Discussion December 2018