Mailman 3 August 2018 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker 11 Jul '23

11 Jul '23

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem 18 Feb '23

18 Feb '23

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell 22 Jul '21

22 Jul '21

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

10 10

Proposal to accept NEP-18, __array_function__ protocol
by Stephan Hoyer 27 Sep '18

27 Sep '18

I propose to accept NEP-18, "A dispatch mechanism for NumPy’s high level array functions": http://www.numpy.org/neps/nep-0018-array-function-protocol.html Since the last round of discussion, we added a new section on "Callable objects generated at runtime" clarifying that to handle such objects is out of scope for the initial proposal in the NEP. If there are no substantive objections within 7 days from this email, then the NEP will be accepted; see NEP 0 for more details. Cheers, Stpehan

12 61

The http://planet.scipy.org/ site is down.
by Charles R Harris 12 Sep '18

12 Sep '18

Hi All, Anyone know who hosts http://planet.scipy.org/? Chuck

6 7

Re: [Numpy-discussion] Adoption of a Code of Conduct
by Peter Creasey 02 Sep '18

02 Sep '18

+1 for keeping the same CoC as Scipy, making a new thing just seems a bigger surface area to maintain. Personally I already assumed Scipy's "honour[ing] diversity in..." did not imply any protection of behaviours that violate the CoC *itself*, but if you wanted to be really explicit you could add "to the extent that these do not conflict with this code of conduct." to that line. Best, Peter

8 13

Possible Deprecation of np.ediff1d
by Tyler Reddy 29 Aug '18

29 Aug '18

Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in. Related discussion on SO: https://stackoverflow.com/questions/39014324/difference-between-numpy-ediff… Thoughts? Best wishes, Tyler

10 13

Copies with MaskedArray?
by Andrew Bell 24 Aug '18

24 Aug '18

Hi, I'm creating a structured numpy array in C, wrapping it with a MaskedArray and passing this to a Python script: ... PyObject *arr = PyArray_NewFromDescr(&PyArray_Type, descr, 1, &longCount, &pointSize, data, NPY_ARRAY_WRITEABLE, nullptr); // Now wrap the created array in a MaskedArray. PyObject *module = PyImport_ImportModule("numpy.ma"); PyObject *dict = PyModule_GetDict(module); PyObject *keys = PyDict_Keys(dict); PyObject *o = PyDict_GetItemString(dict, "MaskedArray"); PyObject *args = PyTuple_New(1); PyTuple_SetItem(args, 0, arr); m_directArray = PyObject_CallObject(o, args); This works fine and the data is seen in python as it was in C. What I'd like to happen is to have changes made to the array in Python be reflected in the memory that was passed in. This seems to work in some cases. Given the dtype: [('X', '<f8'), ('Y', '<f8'), ('Z', '<f8'), ('OffsetTime', '<u4')] I can do something like: X = 10 in Python and appropriate data is set in the C-allocated memory. However, this fails: X *= 10 in that it seems to make a copy of the array IF the mask of the values aren't all set to 0 in the source. Is there something I can do to get operations on the MaskedArray to operate on the data in place as would occur with a raw ndarray? Thanks, -- Andrew Bell andrew.bell.ia(a)gmail.com

1 0

polynomial TODOs and enhancements
by Jeffrey Yancey 24 Aug '18

24 Aug '18

Hi everyone, I spent some time going through the polynomial directory and found several locations where there were comments left suggestion potential improvements down the road, TODOs, and a line of commented out code. I wanted to share these here to gather a consensus on which, if any, of these tasks should be done. If some of these should not be done, do we want to remove the comments implying changes should be made, or kick the can down the road? In `numpy/polynomial/_polybase.py, there is the following class method: @staticmethod def _repr_latex_scalar(x): # TODO: we're stuck with disabling math formatting until we handle # exponents in this function return r'\text{{{}}}'.format(x) Further down, in `_repr_latex_` there is this comprehension, where the leading comment is no longer true, and the old conditional has been commented out: # filter out uninteresting coefficients filtered_coeffs = [ (i, c) for i, c in enumerate(self.coef) # if not (c == 0) # handle NaN ] If this behavior is correct, shouldn’t we remove the leading comment and the commented out line, or at least leave an explanation as to why it is commented out and we wish it to remain. Further down there is also a comment in the `__div__` function implying there may have been the intention of future modification: def __div__(self, other): # set to __floordiv__, /, for now. return self.__floordiv__(other) There is also a comment, “made more efficient by using powers of two in the usual way” in the various `pow` functions specified below: chebyshev.py, line 862 hermite_e.py, line 679 hermite.py, line 630 laguerre.py, line 627 legendre.py, line 661 polynomial.py, line 471 I am happy to make any or all of the enhancements, modify the documentation, or review PRs. I appreciate advice or direction on any of these issues. Jeffrey Yancey

1 0

NumPy 1.15.1 released.
by Charles R Harris 22 Aug '18

22 Aug '18

Hi All, On behalf of the NumPy team, I am pleased to announce the release of NumPy 1.15.1. This is a bugfix release for bugs and regressions reported following the 1.15.0 release. Noticeable fixes are - The annoying but harmless RuntimeWarning that "numpy.dtype size changed" has been suppressed. The long standing suppression was lost in the transition to pytest. - The update to Cython 0.28.3 exposed a problematic use of a gcc attribute used to prefer code size over speed in module initialization, possibly resulting in incorrect compiled code. This has been fixed in latest Cython but has been disabled here for safety. - Support for big-endian and ARMv8 architectures has been improved. The Python versions supported by this release are 2.7, 3.4-3.7. The wheels are linked with OpenBLAS v0.3.0, which should fix some of the linalg problems reported for NumPy 1.14. Wheels for this release can be downloaded from PyPI <https://pypi.org/project/numpy/1.15.1/>, source archives are available from Github <https://github.com/numpy/numpy/releases/tag/v1.15.1>. *Compatibility Note* The NumPy 1.15.x OS X wheels released on PyPI no longer contain 32-bit binaries. That will also be the case in future releases. See #11625 <https://github.com/numpy/numpy/issues/11625> for the related discussion. Those needing 32-bit support on the Mac should look elsewhere or build from source. *Contributors* A total of 7 people contributed to this release. People with a "+" by their names contributed a patch for the first time. * Charles Harris * Chris Billington * Elliott Sales de Andrade + * Eric Wieser * Jeremy Manning + * Matti Picus * Ralf Gommers *Pull requests merged* A total of 24 pull requests were merged for this release. * `#11647 <https://github.com/numpy/numpy/pull/11647>`__: MAINT: Filter Cython warnings in ``__init__.py`` * `#11648 <https://github.com/numpy/numpy/pull/11648>`__: BUG: Fix doc source links to unwrap decorators * `#11657 <https://github.com/numpy/numpy/pull/11657>`__: BUG: Ensure singleton dimensions are not dropped when converting... * `#11661 <https://github.com/numpy/numpy/pull/11661>`__: BUG: Warn on Nan in minimum,maximum for scalars * `#11665 <https://github.com/numpy/numpy/pull/11665>`__: BUG: cython sometimes emits invalid gcc attribute * `#11682 <https://github.com/numpy/numpy/pull/11682>`__: BUG: Fix regression in void_getitem * `#11698 <https://github.com/numpy/numpy/pull/11698>`__: BUG: Make matrix_power again work for object arrays. * `#11700 <https://github.com/numpy/numpy/pull/11700>`__: BUG: Add missing PyErr_NoMemory after failing malloc * `#11719 <https://github.com/numpy/numpy/pull/11719>`__: BUG: Fix undefined functions on big-endian systems. * `#11720 <https://github.com/numpy/numpy/pull/11720>`__: MAINT: Make einsum optimize default to False. * `#11746 <https://github.com/numpy/numpy/pull/11746>`__: BUG: Fix regression in loadtxt for bz2 text files in Python 2. * `#11757 <https://github.com/numpy/numpy/pull/11757>`__: BUG: Revert use of `console_scripts`. * `#11758 <https://github.com/numpy/numpy/pull/11758>`__: BUG: Fix Fortran kind detection for aarch64 & s390x. * `#11759 <https://github.com/numpy/numpy/pull/11759>`__: BUG: Fix printing of longdouble on ppc64le. * `#11760 <https://github.com/numpy/numpy/pull/11760>`__: BUG: Fixes for unicode field names in Python 2 * `#11761 <https://github.com/numpy/numpy/pull/11761>`__: BUG: Increase required cython version on python 3.7 * `#11763 <https://github.com/numpy/numpy/pull/11763>`__: BUG: check return value of _buffer_format_string * `#11775 <https://github.com/numpy/numpy/pull/11775>`__: MAINT: Make assert_array_compare more generic. * `#11776 <https://github.com/numpy/numpy/pull/11776>`__: TST: Fix urlopen stubbing. * `#11777 <https://github.com/numpy/numpy/pull/11777>`__: BUG: Fix regression in intersect1d. * `#11779 <https://github.com/numpy/numpy/pull/11779>`__: BUG: Fix test sensitive to platform byte order. * `#11781 <https://github.com/numpy/numpy/pull/11781>`__: BUG: Avoid signed overflow in histogram * `#11785 <https://github.com/numpy/numpy/pull/11785>`__: BUG: Fix pickle and memoryview for datetime64, timedelta64 scalars * `#11786 <https://github.com/numpy/numpy/pull/11786>`__: BUG: Deprecation triggers segfault Cheers, Charles Harris

1 0