Mailman 3 October 2013 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker July 11, 2023

July 11, 2023

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem Feb. 18, 2023

Feb. 18, 2023

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell July 22, 2021

July 22, 2021

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

10 10

C-API: multidimensional array indexing?
by Johann Bauer March 31, 2016

March 31, 2016

Dear experts, is there a C-API function for numpy which implements Python's multidimensional indexing? Say, I have a 2d-array PyArrayObject * M; and an index int i; how do I extract the i-th row or column M[i,:] respectively M[:,i]? I am looking for a function which gives again a PyArrayObject * and which is a view to M (no copied data; the result should be another PyArrayObject whose data and strides points to the correct memory portion of M). I searched the API documentation, Google and mailing lists for quite a long time but didn't find anything. Can you help me? Thanks, Johann

3 3

taking a 2D uneven surface slice
by Bryan Woods Dec. 4, 2014

Dec. 4, 2014

I'm trying to do something that at first glance I think should be simple but I can't quite figure out how to do it. The problem is as follows: I have a 3D grid Values[Nx, Ny, Nz] I want to slice Values at a 2D surface in the Z dimension specified by Z_index[Nx, Ny] and return a 2D slice[Nx, Ny]. It is not as simple as Values[:,:,Z_index]. I tried this: >>> values.shape (4, 5, 6) >>> coords.shape (4, 5) >>> slice = values[:,:,coords] >>> slice.shape (4, 5, 4, 5) >>> slice = np.take(values, coords, axis=2) >>> slice.shape (4, 5, 4, 5) >>> Obviously I could create an empty 2D slice and then fill it by using np.ndenumerate to fill it point by point by selecting values[i, j, Z_index[i, j]]. This just seems too inefficient and not very pythonic.

3 3

Speedup by avoiding memory alloc twice in scalar array
by Arink Verma Jan. 8, 2014

Jan. 8, 2014

Hi, I am working on performance parity between numpy scalar/small array and python array as GSOC mentored By Charles. Currently I am looking at PyArray_Return, which allocate separate memory just for scalar return. Unlike python which allocate memory once for returning result of scalar operations; numpy calls malloc twice once for the array object itself, and a second time for the array data. These memory allocations are happening in PyArray_NewFromDescr and PyArray_Scalar. Stashing both within a single allocation would be more efficient. In, PyArray_Scalar, new struct (PyLongScalarObject) need allocation in case of scalar arrays. Instead, can we just some how convert/cast PyArrayObject to PyLongScalarObject.?? -- Arink Verma www.arinkverma.in

5 12

RFC: is it worth giving a lightning talk at PyCon 2014 on numpy vbench-marking?
by Yaroslav Halchenko Nov. 26, 2013

Nov. 26, 2013

Hi Guys, PyCon 2014 will be just around the corner from where I am, so I decided to attend. Being lazy (or busy) I haven't submitted any big talk but thinking to submit few lightning talks (just 5 min and 400 characters abstract limit), and I think it might be worth letting people know about my little project. I would really appreciate your sincere feedback (e.g. "not worth it" would be valuable too). Here is the title/abstract numpy-vbench -- speed benchmarks for NumPy http://yarikoptic.github.io/numpy-vbench provides collection of speed performance benchmarks for NumPy. Benchmarking of multiple maintenance and current development branches allows not only to timely react to new performance regressions, but also to compare NumPy performance across releases. Your contributions would help to guarantee that your code does not become slower with a new NumPy release. btw -- fresh results are here http://yarikoptic.github.io/numpy-vbench/ . I have tuned benchmarking so it now reflects the best performance across multiple executions of the whole battery, thus eliminating spurious variance if estimate is provided from a single point in time. Eventually I expect many of those curves to become even "cleaner". -- Yaroslav O. Halchenko, Ph.D. http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Senior Research Associate, Psychological and Brain Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik

6 13

Announcing Bokeh 0.2: interactive web plotting for Python
by Peter Wang Nov. 7, 2013

Nov. 7, 2013

Hi everyone, I'm excited to announce the v0.2 release of Bokeh, an interactive web plotting library for Python. The long-term vision for Bokeh is to provide rich interactivity, using the full power of Javascript and Canvas, to Python users who don't need to write any JS or learn the DOM. The full blog post announcement is here: http://continuum.io/blog/bokeh02 The project website (with interactive gallery) is at: http://bokeh.pydata.org And the Git repo is: https://github.com/ContinuumIO/bokeh Cheers, Peter

3 10

ANN: NumPy 1.8.0 release.
by Charles R Harris Nov. 3, 2013

Nov. 3, 2013

I am pleased to announce the availability of NumPy 1.8.0. This release is the culmination of over a years worth of work by the NumPy team and contains many fixes, enhancements, and new features. Highlights are: - New, no 2to3, Python 2 and Python 3 are supported by a common code base. - New, gufuncs for linear algebra, enabling operations on stacked arrays. - New, inplace fancy indexing for ufuncs with the ``.at`` method. - New, ``partition`` function, partial sorting via selection for fast median. - New, ``nanmean``, ``nanvar``, and ``nanstd`` functions skipping NaNs. - New, ``full`` and ``full_like`` functions to create value initialized arrays. - New, ``PyUFunc_RegisterLoopForDescr``, better ufunc support for user dtypes. - Numerous performance improvements in many areas. This release requires Python 2.6, 2.7 or 3.2-3.3, support for Python 2.4 and 2.5 has been dropped. Sources and binaries can be found at http://sourceforge.net/projects/numpy/files/NumPy/1.8.0/. Some 119 people contributed to this release. This is a remarkable increase and shows that there is still life in this venerable code that had its beginning in Numeric some 18 years ago. Many thanks to you all. Enjoy, Chuck NumPy 1.8.0 Release Notes ************************* This release supports Python 2.6 -2.7 and 3.2 - 3.3. Highlights ========== * New, no 2to3, Python 2 and Python 3 are supported by a common code base. * New, gufuncs for linear algebra, enabling operations on stacked arrays. * New, inplace fancy indexing for ufuncs with the ``.at`` method. * New, ``partition`` function, partial sorting via selection for fast median. * New, ``nanmean``, ``nanvar``, and ``nanstd`` functions skipping NaNs. * New, ``full`` and ``full_like`` functions to create value initialized arrays. * New, ``PyUFunc_RegisterLoopForDescr``, better ufunc support for user dtypes. * Numerous performance improvements in many areas. Dropped Support =============== Support for Python versions 2.4 and 2.5 has been dropped, Support for SCons has been removed. Future Changes ============== The Datetime64 type remains experimental in this release. In 1.9 there will probably be some changes to make it more useable. The diagonal method currently returns a new array and raises a FutureWarning. In 1.9 it will return a readonly view. Multiple field selection from a array of structured type currently returns a new array and raises a FutureWarning. In 1.9 it will return a readonly view. The numpy/oldnumeric and numpy/numarray compatibility modules will be removed in 1.9. Compatibility notes =================== The doc/sphinxext content has been moved into its own github repository, and is included in numpy as a submodule. See the instructions in doc/HOWTO_BUILD_DOCS.rst.txt for how to access the content. .. _numpydoc: https://github.com/numpy/numpydoc The hash function of numpy.void scalars has been changed. Previously the pointer to the data was hashed as an integer. Now, the hash function uses the tuple-hash algorithm to combine the hash functions of the elements of the scalar, but only if the scalar is read-only. Numpy has switched its build system to using 'separate compilation' by default. In previous releases this was supported, but not default. This should produce the same results as the old system, but if you're trying to do something complicated like link numpy statically or using an unusual compiler, then it's possible you will encounter problems. If so, please file a bug and as a temporary workaround you can re-enable the old build system by exporting the shell variable NPY_SEPARATE_COMPILATION=0. For the AdvancedNew iterator the ``oa_ndim`` flag should now be -1 to indicate that no ``op_axes`` and ``itershape`` are passed in. The ``oa_ndim == 0`` case, now indicates a 0-D iteration and ``op_axes`` being NULL and the old usage is deprecated. This does not effect the ``NpyIter_New`` or ``NpyIter_MultiNew`` functions. The functions nanargmin and nanargmax now return np.iinfo['intp'].min for the index in all-NaN slices. Previously the functions would raise a ValueError for array returns and NaN for scalar returns. NPY_RELAXED_STRIDES_CHECKING ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ There is a new compile time environment variable ``NPY_RELAXED_STRIDES_CHECKING``. If this variable is set to 1, then numpy will consider more arrays to be C- or F-contiguous -- for example, it becomes possible to have a column vector which is considered both C- and F-contiguous simultaneously. The new definition is more accurate, allows for faster code that makes fewer unnecessary copies, and simplifies numpy's code internally. However, it may also break third-party libraries that make too-strong assumptions about the stride values of C- and F-contiguous arrays. (It is also currently known that this breaks Cython code using memoryviews, which will be fixed in Cython.) THIS WILL BECOME THE DEFAULT IN A FUTURE RELEASE, SO PLEASE TEST YOUR CODE NOW AGAINST NUMPY BUILT WITH:: NPY_RELAXED_STRIDES_CHECKING=1 python setup.py install You can check whether NPY_RELAXED_STRIDES_CHECKING is in effect by running:: np.ones((10, 1), order="C").flags.f_contiguous This will be ``True`` if relaxed strides checking is enabled, and ``False`` otherwise. The typical problem we've seen so far is C code that works with C-contiguous arrays, and assumes that the itemsize can be accessed by looking at the last element in the ``PyArray_STRIDES(arr)`` array. When relaxed strides are in effect, this is not true (and in fact, it never was true in some corner cases). Instead, use ``PyArray_ITEMSIZE(arr)``. For more information check the "Internal memory layout of an ndarray" section in the documentation. Binary operations with non-arrays as second argument ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Binary operations of the form ``<array-or-subclass> * <non-array-subclass>`` where ``<non-array-subclass>`` declares an ``__array_priority__`` higher than that of ``<array-or-subclass>`` will now unconditionally return *NotImplemented*, giving ``<non-array-subclass>`` a chance to handle the operation. Previously, `NotImplemented` would only be returned if ``<non-array-subclass>`` actually implemented the reversed operation, and after a (potentially expensive) array conversion of ``<non-array-subclass>`` had been attempted. (`bug <https://github.com/numpy/numpy/issues/3375>`_, `pull request <https://github.com/numpy/numpy/pull/3501>`_) Function `median` used with `overwrite_input` only partially sorts array ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If `median` is used with `overwrite_input` option the input array will now only be partially sorted instead of fully sorted. Fix to financial.npv ~~~~~~~~~~~~~~~~~~~~ The npv function had a bug. Contrary to what the documentation stated, it summed from indexes ``1`` to ``M`` instead of from ``0`` to ``M - 1``. The fix changes the returned value. The mirr function called the npv function, but worked around the problem, so that was also fixed and the return value of the mirr function remains unchanged. Runtime warnings when comparing NaN numbers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Comparing ``NaN`` floating point numbers now raises the ``invalid`` runtime warning. If a ``NaN`` is expected the warning can be ignored using np.errstate. E.g.:: with np.errstate(invalid='ignore'): operation() New Features ============ Support for linear algebra on stacked arrays ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The gufunc machinery is now used for np.linalg, allowing operations on stacked arrays and vectors. For example:: >>> a array([[[ 1., 1.], [ 0., 1.]], [[ 1., 1.], [ 0., 1.]]]) >>> np.linalg.inv(a) array([[[ 1., -1.], [ 0., 1.]], [[ 1., -1.], [ 0., 1.]]]) In place fancy indexing for ufuncs ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The function ``at`` has been added to ufunc objects to allow in place ufuncs with no buffering when fancy indexing is used. For example, the following will increment the first and second items in the array, and will increment the third item twice: ``numpy.add.at(arr, [0, 1, 2, 2], 1)`` This is what many have mistakenly thought ``arr[[0, 1, 2, 2]] += 1`` would do, but that does not work as the incremented value of ``arr[2]`` is simply copied into the third slot in ``arr`` twice, not incremented twice. New functions `partition` and `argpartition` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ New functions to partially sort arrays via a selection algorithm. A ``partition`` by index ``k`` moves the ``k`` smallest element to the front of an array. All elements before ``k`` are then smaller or equal than the value in position ``k`` and all elements following ``k`` are then greater or equal than the value in position ``k``. The ordering of the values within these bounds is undefined. A sequence of indices can be provided to sort all of them into their sorted position at once iterative partitioning. This can be used to efficiently obtain order statistics like median or percentiles of samples. ``partition`` has a linear time complexity of ``O(n)`` while a full sort has ``O(n log(n))``. New functions `nanmean`, `nanvar` and `nanstd` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ New nan aware statistical functions are added. In these functions the results are what would be obtained if nan values were ommited from all computations. New functions `full` and `full_like` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ New convenience functions to create arrays filled with a specific value; complementary to the existing `zeros` and `zeros_like` functions. IO compatibility with large files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Large NPZ files >2GB can be loaded on 64-bit systems. Building against OpenBLAS ~~~~~~~~~~~~~~~~~~~~~~~~~ It is now possible to build numpy against OpenBLAS by editing site.cfg. New constant ~~~~~~~~~~~~ Euler's constant is now exposed in numpy as euler_gamma. New modes for qr ~~~~~~~~~~~~~~~~ New modes 'complete', 'reduced', and 'raw' have been added to the qr factorization and the old 'full' and 'economic' modes are deprecated. The 'reduced' mode replaces the old 'full' mode and is the default as was the 'full' mode, so backward compatibility can be maintained by not specifying the mode. The 'complete' mode returns a full dimensional factorization, which can be useful for obtaining a basis for the orthogonal complement of the range space. The 'raw' mode returns arrays that contain the Householder reflectors and scaling factors that can be used in the future to apply q without needing to convert to a matrix. The 'economic' mode is simply deprecated, there isn't much use for it and it isn't any more efficient than the 'raw' mode. New `invert` argument to `in1d` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The function `in1d` now accepts a `invert` argument which, when `True`, causes the returned array to be inverted. Advanced indexing using `np.newaxis` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It is now possible to use `np.newaxis`/`None` together with index arrays instead of only in simple indices. This means that ``array[np.newaxis, [0, 1]]`` will now work as expected and select the first two rows while prepending a new axis to the array. C-API ~~~~~ New ufuncs can now be registered with builtin input types and a custom output type. Before this change, NumPy wouldn't be able to find the right ufunc loop function when the ufunc was called from Python, because the ufunc loop signature matching logic wasn't looking at the output operand type. Now the correct ufunc loop is found, as long as the user provides an output argument with the correct output type. runtests.py ~~~~~~~~~~~ A simple test runner script ``runtests.py`` was added. It also builds Numpy via ``setup.py build`` and can be used to run tests easily during development. Improvements ============ IO performance improvements ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Performance in reading large files was improved by chunking (see also IO compatibility). Performance improvements to `pad` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The `pad` function has a new implementation, greatly improving performance for all inputs except `mode=<function>` (retained for backwards compatibility). Scaling with dimensionality is dramatically improved for rank >= 4. Performance improvements to `isnan`, `isinf`, `isfinite` and `byteswap` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `isnan`, `isinf`, `isfinite` and `byteswap` have been improved to take advantage of compiler builtins to avoid expensive calls to libc. This improves performance of these operations by about a factor of two on gnu libc systems. Performance improvements via SSE2 vectorization ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Several functions have been optimized to make use of SSE2 CPU SIMD instructions. * Float32 and float64: * base math (`add`, `subtract`, `divide`, `multiply`) * `sqrt` * `minimum/maximum` * `absolute` * Bool: * `logical_or` * `logical_and` * `logical_not` This improves performance of these operations up to 4x/2x for float32/float64 and up to 10x for bool depending on the location of the data in the CPU caches. The performance gain is greatest for in-place operations. In order to use the improved functions the SSE2 instruction set must be enabled at compile time. It is enabled by default on x86_64 systems. On x86_32 with a capable CPU it must be enabled by passing the appropriate flag to the CFLAGS build variable (-msse2 with gcc). Performance improvements to `median` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `median` is now implemented in terms of `partition` instead of `sort` which reduces its time complexity from O(n log(n)) to O(n). If used with the `overwrite_input` option the array will now only be partially sorted instead of fully sorted. Overrideable operand flags in ufunc C-API ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When creating a ufunc, the default ufunc operand flags can be overridden via the new op_flags attribute of the ufunc object. For example, to set the operand flag for the first input to read/write: PyObject \*ufunc = PyUFunc_FromFuncAndData(...); ufunc->op_flags[0] = NPY_ITER_READWRITE; This allows a ufunc to perform an operation in place. Also, global nditer flags can be overridden via the new iter_flags attribute of the ufunc object. For example, to set the reduce flag for a ufunc: ufunc->iter_flags = NPY_ITER_REDUCE_OK; Changes ======= General ~~~~~~~ The function np.take now allows 0-d arrays as indices. The separate compilation mode is now enabled by default. Several changes to np.insert and np.delete: * Previously, negative indices and indices that pointed past the end of the array were simply ignored. Now, this will raise a Future or Deprecation Warning. In the future they will be treated like normal indexing treats them -- negative indices will wrap around, and out-of-bound indices will generate an error. * Previously, boolean indices were treated as if they were integers (always referring to either the 0th or 1st item in the array). In the future, they will be treated as masks. In this release, they raise a FutureWarning warning of this coming change. * In Numpy 1.7. np.insert already allowed the syntax `np.insert(arr, 3, [1,2,3])` to insert multiple items at a single position. In Numpy 1.8. this is also possible for `np.insert(arr, [3], [1, 2, 3])`. Padded regions from np.pad are now correctly rounded, not truncated. C-API Array Additions ~~~~~~~~~~~~~~~~~~~~~ Four new functions have been added to the array C-API. * PyArray_Partition * PyArray_ArgPartition * PyArray_SelectkindConverter * PyDataMem_NEW_ZEROED C-API Ufunc Additions ~~~~~~~~~~~~~~~~~~~~~ One new function has been added to the ufunc C-API that allows to register an inner loop for user types using the descr. * PyUFunc_RegisterLoopForDescr Deprecations ============ The 'full' and 'economic' modes of qr factorization are deprecated. General ~~~~~~~ The use of non-integer for indices and most integer arguments has been deprecated. Previously float indices and function arguments such as axes or shapes were truncated to integers without warning. For example `arr.reshape(3., -1)` or `arr[0.]` will trigger a deprecation warning in NumPy 1.8., and in some future version of NumPy they will raise an error. Authors ======= This release contains work by the following people who contributed at least one patch to this release. The names are in alphabetical order by first name: * 87 * Adam Ginsburg + * Adam Griffiths + * Alexander Belopolsky + * Alex Barth + * Alex Ford + * Andreas Hilboll + * Andreas Kloeckner + * Andreas Schwab + * Andrew Horton + * argriffing + * Arink Verma + * Bago Amirbekian + * Bartosz Telenczuk + * bebert218 + * Benjamin Root + * Bill Spotz + * Bradley M. Froehle * Carwyn Pelley + * Charles Harris * Chris * Christian Brueffer + * Christoph Dann + * Christoph Gohlke * Dan Hipschman + * Daniel + * Dan Miller + * daveydave400 + * David Cournapeau * David Warde-Farley * Denis Laxalde * dmuellner + * Edward Catmur + * Egor Zindy + * endolith * Eric Firing * Eric Fode * Eric Moore + * Eric Price + * Fazlul Shahriar + * Félix Hartmann + * Fernando Perez * Frank B + * Frank Breitling + * Frederic * Gabriel * GaelVaroquaux * Guillaume Gay + * Han Genuit * HaroldMills + * hklemm + * jamestwebber + * Jason Madden + * Jay Bourque * jeromekelleher + * Jesús Gómez + * jmozmoz + * jnothman + * Johannes Schönberger + * John Benediktsson + * John Salvatier + * John Stechschulte + * Jonathan Waltman + * Joon Ro + * Jos de Kloe + * Joseph Martinot-Lagarde + * Josh Warner (Mac) + * Jostein Bø Fløystad + * Juan Luis Cano Rodríguez + * Julian Taylor + * Julien Phalip + * K.-Michael Aye + * Kumar Appaiah + * Lars Buitinck * Leon Weber + * Luis Pedro Coelho * Marcin Juszkiewicz * Mark Wiebe * Marten van Kerkwijk + * Martin Baeuml + * Martin Spacek * Martin Teichmann + * Matt Davis + * Matthew Brett * Maximilian Albert + * m-d-w + * Michael Droettboom * mwtoews + * Nathaniel J. Smith * Nicolas Scheffer + * Nils Werner + * ochoadavid + * Ondřej Čertík * ovillellas + * Paul Ivanov * Pauli Virtanen * peterjc * Ralf Gommers * Raul Cota + * Richard Hattersley + * Robert Costa + * Robert Kern * Rob Ruana + * Ronan Lamy * Sandro Tosi * Sascha Peilicke + * Sebastian Berg * Skipper Seabold * Stefan van der Walt * Steve + * Takafumi Arakaki + * Thomas Robitaille + * Tomas Tomecek + * Travis E. Oliphant * Valentin Haenel * Vladimir Rutsky + * Warren Weckesser * Yaroslav Halchenko * Yury V. Zaytsev + A total of 119 people contributed to this release. People with a "+" by their names contributed a patch for the first time.

4 7

char with native integer signedness
by Geoffrey Irving Nov. 1, 2013

Nov. 1, 2013

Is there a standard way in numpy of getting a char with C-native integer signedness? I.e., boost::is_signed<char>::value ? numpy.byte : numpy.ubyte but without nonsensical mixing of languages? Thanks, Geoffrey

3 4