Mailman 3 October 2015 - SciPy-Dev

scipy.integrate.nquad full_output
by Nathan Woods Oct. 9, 2015

Oct. 9, 2015

Hey everyone, I need a design consult. I'm trying to enable the use of the quad full_output option when using nquad, and I'm trying to come up with a good way to manage all of the data that comes back from using something like that. Any suggestions are welcome. https://github.com/scipy/scipy/issues/5330 Nathan Woods

1 1

Re: [SciPy-Dev] Boundary Value Problem solver for inclusion in SciPy
by David Pugh Oct. 8, 2015

Oct. 8, 2015

Excellent. The current design of my API separates solving a BVP into three parts. 1. specify your problem (similar to scikits.bvp_solver) 2. choose your basis functions (currently, any of the polynomial classes from NumPy are supported, in very near future I plan to add B-splines, in medium term I would hope to add support for a shape-preserving option) 3. solve (solvers just wrap underlying nonlinear equations solvers in SciPy) I have a number of example notebooks <https://github.com/davidrpugh/pyCollocation/tree/master/examples> and would welcome pointers BVPs that would be useful for either examples or additional tests. D On Thu, Oct 8, 2015 at 1:00 PM, <scipy-dev-request(a)scipy.org> wrote: > Send SciPy-Dev mailing list submissions to > scipy-dev(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.scipy.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request(a)scipy.org > > You can reach the person managing the list at > scipy-dev-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-Dev digest..." > > > Today's Topics: > > 1. scipy.integrate.nquad full_output (Nathan Woods) > 2. Re: vectorized scipy.integrate.quad (Anne Archibald) > 3. Re: Boundary Value Problem solver for inclusion in SciPy > (Gyro Funch) > 4. Re: Boundary Value Problem solver for inclusion in SciPy > (Pauli Virtanen) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 7 Oct 2015 09:44:30 -0600 > From: Nathan Woods <charlesnwoods(a)gmail.com> > To: SciPy Developers List <scipy-dev(a)scipy.org> > Subject: [SciPy-Dev] scipy.integrate.nquad full_output > Message-ID: > < > CAAYHtCSu+7bJ-HWKFKsN-oPRByws4w+6NiUS3LEY-2_rrhrEzQ(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hey everyone, > > I need a design consult. I'm trying to enable the use of the quad > full_output option when using nquad, and I'm trying to come up with a good > way to manage all of the data that comes back from using something like > that. Any suggestions are welcome. > > https://github.com/scipy/scipy/issues/5330 > > Nathan Woods >

1 0

Boundary Value Problem solver for inclusion in SciPy
by David Pugh Oct. 8, 2015

Oct. 8, 2015

All, I am developing a pure-python (i.e., no external fortran/C dependencies) two-point boundary value solver tentatively called pyCollocation <https://github.com/davidrpugh/pyCollocation> and I am writing to gauge the level of interest in eventually including the solver in SciPy. v/r, Dr. David R. Pugh Post-doctoral research fellow INET Oxford Mathematical Institute Oxford University On Wed, Oct 7, 2015 at 1:00 PM, <scipy-dev-request(a)scipy.org> wrote: > Send SciPy-Dev mailing list submissions to > scipy-dev(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.scipy.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request(a)scipy.org > > You can reach the person managing the list at > scipy-dev-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-Dev digest..." > > > Today's Topics: > > 1. Re: [Numpy-discussion] Numpy 1.10.0 release (Da?id) > 2. Pull Request #5318 (Henry Herbol) > 3. Scipy.io.savemat optimization issue. (Jamie Tsao) > 4. Re: [Numpy-discussion] Numpy 1.10.0 release (Julian Taylor) > 5. Re: [Numpy-discussion] Numpy 1.10.0 release (Julian Taylor) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 6 Oct 2015 14:08:36 +0200 > From: Da?id <davidmenhur(a)gmail.com> > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Cc: SciPy Users List <scipy-user(a)scipy.org>, SciPy Developers List > <scipy-dev(a)scipy.org> > Subject: Re: [SciPy-Dev] [Numpy-discussion] Numpy 1.10.0 release > Message-ID: > <CAJhcF= > 1PDQQUukR4Fdby+1ygsJeLNZRFyRVFYVAzDhueAgM06w(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > I don't get any failures on Fedora 22. I have installed it with pip, > setting my CFLAGS to "-march=core-avx-i -O2 -pipe -mtune=native" and > linking against openblas. > > With the new Numpy, Scipy full suite shows two errors, I am sorry I didn't > think of running that in the RC phase: > > ====================================================================== > FAIL: test_weighting (test_stats.TestHistogram) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > > "/home/david/.local/virtualenv/py27/lib/python2.7/site-packages/scipy/stats/tests/test_stats.py", > line 892, in test_weighting > decimal=2) > File > > "/home/david/.local/virtualenv/py27/lib/python2.7/site-packages/numpy/testing/utils.py", > line 886, in assert_array_almost_equal > precision=decimal) > File > > "/home/david/.local/virtualenv/py27/lib/python2.7/site-packages/numpy/testing/utils.py", > line 708, in assert_array_compare > raise AssertionError(msg) > AssertionError: > Arrays are not almost equal to 2 decimals > > (mismatch 40.0%) > x: array([ 4. , 0. , 4.5, -0.9, 0. , 0.3, 110.2, 0. , > 0. , 42. ]) > y: array([ 4. , 0. , 4.5, -0.9, 0.3, 0. , 7. , 103.2, > 0. , 42. ]) > > ====================================================================== > FAIL: test_nanmedian_all_axis (test_stats.TestNanFunc) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > > "/home/david/.local/virtualenv/py27/lib/python2.7/site-packages/scipy/stats/tests/test_stats.py", > line 226, in test_nanmedian_all_axis > assert_equal(len(w), 4) > File > > "/home/david/.local/virtualenv/py27/lib/python2.7/site-packages/numpy/testing/utils.py", > line 354, in assert_equal > raise AssertionError(msg) > AssertionError: > Items are not equal: > ACTUAL: 1 > DESIRED: 4 > > I am almost sure these errors weren't there before. > > On 6 October 2015 at 13:53, Neal Becker <ndbecker2(a)gmail.com> wrote: > > > 1 test failure: > > > > FAIL: test_blasdot.test_blasdot_used > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in > > runTest > > self.test(*self.arg) > > File "/home/nbecker/.local/lib/python2.7/site- > > packages/numpy/testing/decorators.py", line 146, in skipper_func > > return f(*args, **kwargs) > > File "/home/nbecker/.local/lib/python2.7/site- > > packages/numpy/core/tests/test_blasdot.py", line 31, in test_blasdot_used > > assert_(dot is _dotblas.dot) > > File "/home/nbecker/.local/lib/python2.7/site- > > packages/numpy/testing/utils.py", line 53, in assert_ > > raise AssertionError(smsg) > > AssertionError > > > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion(a)scipy.org > > https://mail.scipy.org/mailman/listinfo/numpy-discussion > > >

3 2

vectorized scipy.integrate.quad
by Christoph Groth Oct. 7, 2015

Oct. 7, 2015

Dear SciPy experts, We would like to speed-up some numerical integrations that we do by parallelizing them with MPI (our integrands take a very long time to evaluate). Looking at the implementation of QAGSE (that is performing the work for scipy.integrate.quad) I see that there is potential for 21-fold vectorization in the first pass and subsequently for 42-fold vectorization of the evaluation of the integrand. That’s quite good! Once a vectorized scipy.integrand.quad is available, any parallelization technique can be used in the integrand can use. But even without parallelization a vectorized integrand could already provide significant speedups. On the Python level, I think that the interface to the new function could remain the same as that of the existing quad routine, with the only difference that the first argument of the integrand will be not a number, but a 1d-array of numbers. Would such a vectorized quad be a welcome addition to SciPy? I imagine proceeding as follows: (1) Download the relevant parts of QUADPACK and convert them to C using f2c. (2) Vectorize the C code. (3) Add Python bindings. (4) Write a test that verifies that the vectorized routine evaluates the same points and gives the same results as the sequential one. Cheers Christoph

5 11

Re: [SciPy-Dev] [Numpy-discussion] Numpy 1.10.0 release
by Daπid Oct. 7, 2015

Oct. 7, 2015

I don't get any failures on Fedora 22. I have installed it with pip, setting my CFLAGS to "-march=core-avx-i -O2 -pipe -mtune=native" and linking against openblas. With the new Numpy, Scipy full suite shows two errors, I am sorry I didn't think of running that in the RC phase: ====================================================================== FAIL: test_weighting (test_stats.TestHistogram) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/david/.local/virtualenv/py27/lib/python2.7/site-packages/scipy/stats/tests/test_stats.py", line 892, in test_weighting decimal=2) File "/home/david/.local/virtualenv/py27/lib/python2.7/site-packages/numpy/testing/utils.py", line 886, in assert_array_almost_equal precision=decimal) File "/home/david/.local/virtualenv/py27/lib/python2.7/site-packages/numpy/testing/utils.py", line 708, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal to 2 decimals (mismatch 40.0%) x: array([ 4. , 0. , 4.5, -0.9, 0. , 0.3, 110.2, 0. , 0. , 42. ]) y: array([ 4. , 0. , 4.5, -0.9, 0.3, 0. , 7. , 103.2, 0. , 42. ]) ====================================================================== FAIL: test_nanmedian_all_axis (test_stats.TestNanFunc) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/david/.local/virtualenv/py27/lib/python2.7/site-packages/scipy/stats/tests/test_stats.py", line 226, in test_nanmedian_all_axis assert_equal(len(w), 4) File "/home/david/.local/virtualenv/py27/lib/python2.7/site-packages/numpy/testing/utils.py", line 354, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 1 DESIRED: 4 I am almost sure these errors weren't there before. On 6 October 2015 at 13:53, Neal Becker <ndbecker2(a)gmail.com> wrote: > 1 test failure: > > FAIL: test_blasdot.test_blasdot_used > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in > runTest > self.test(*self.arg) > File "/home/nbecker/.local/lib/python2.7/site- > packages/numpy/testing/decorators.py", line 146, in skipper_func > return f(*args, **kwargs) > File "/home/nbecker/.local/lib/python2.7/site- > packages/numpy/core/tests/test_blasdot.py", line 31, in test_blasdot_used > assert_(dot is _dotblas.dot) > File "/home/nbecker/.local/lib/python2.7/site- > packages/numpy/testing/utils.py", line 53, in assert_ > raise AssertionError(smsg) > AssertionError > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion >

2 1

Re: [SciPy-Dev] [Numpy-discussion] Numpy 1.10.0 release
by Julian Taylor Oct. 7, 2015

Oct. 7, 2015

On 10/06/2015 01:45 PM, Neal Becker wrote: > Are extra_compile_args actually used in all compile steps? extra_compile_args is not used by numpy, its to support some third party use case I never understood. As the typical site.cfg used by numpy only contains binaries that are never compiled by numpy itself it should have no effect on anything. > CFLAGS='-march=native -O3' python setup.py build > > Does seem to use my CFLAGS, as it always did on previous numpy versions. > still seems to work for me, though the preferred variable is OPT= as CFLAGS will contain a bunch of other stuff related to building python extensions themselves (e.g. -fno-strict-aliasing)

1 0

Scipy.io.savemat optimization issue.
by Jamie Tsao Oct. 7, 2015

Oct. 7, 2015

I was attempting to add an optimization to help save some memory from using savemat along with my other commits attempting to fix a bug concerning compression with zlib. The optimization uses an np.array's data attribute, which in python 2.7 is a buffer. This is nice, because in the case the array is Fortran contiguous (much so with 1D arrays unless created from .real or .imag from complex arrays), then I can just pass the data buffer to file.write() without using much memory. I.e. a real sparse matrix will at most use 133% of the matrices memory to save it to disk. And even if it isn't Fortran compliant, I would just do as originally done: use tostring() to achieve the bytes in Fortran order. But then comes python 3. Using python 3.4, I found that now np.array.data is not a buffer but a memoryview. Unfortunately, the memoryview doesn't have the same ability to grab the underlying bytes in the same manner, so file.write() won't write it correctly. Furthermore, file.write() only accepts strings, not bytes. Hence, I would have to do something like str(bytes(arr.data)) to pass it and save to disk, which isn't as good as calling tostring(). What should I do to get around this? My failed pull request is here, where the last commit concerns this problem: http://www.github.com/scipy/scipy/pull/5325 On a side note, is my approach to byte counting fine? I was hoping that it will no longer need to seek around, which helps out a lot with compression, but my current code does this even without compression. Originally, I was afraid the running time would be twice as long (although I claim not many will repeatedly use savemat to the point of it being a bottleneck), but it turns out its runtime is nearly the same. Weird? Lastly, I didn't add any test cases. The only bug I fixed was concerning compressing down a >2GB (in my case, sparse) matrix, but I don't want test cases to create massive matrices and use up tons of memory. -Jamie Tsao Sent from Yahoo Mail on Android

1 0

Pull Request #5318
by Henry Herbol Oct. 6, 2015

Oct. 6, 2015

To Whomever is Interested, I have submitted a Pull Request for an optimization (minimization) method. It involves an adaptation of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm in which the line search step is skipped. This was done because there are times in which optimization is required independent of the target function, and there was no code in the current Scipy.optimize methods that accomodated this. The method employed took advantage of the BFGS(Hess) algorithm touched on by Sheppard et al. (ref here <http://theory.cm.utexas.edu/henkelman/pubs/sheppard08_134106.pdf>) and adjusts the step size of alpha (by a user defined beta parameter) instead of maintaining a small alpha. Henry Herbol P.S. If this was not the way I was supposed to address the scipy-dev mailing list as described here <http://docs.scipy.org/doc/scipy/reference/hacking.html>, I'm sorry for misusing the e-mail. -- Contact E-mail: hherbol(a)gmail.com

1 0

Bug in place_poles
by Irvin Probst Oct. 6, 2015

Oct. 6, 2015

Hi, FYI there is a bug in place_poles, the return values rtol and nb_iter are supposed to be np.nan when there is nothing to optimize and 0 when the solution is unique but the function does it the other way around. Note that the doc contains an error too, it is stated: rtol will be NaN if the optimisation algorithms can not run, i.e when B.shape[1] == 1, or 0 when the solution is unique. But one should read: rtol will be NaN if the optimisation algorithms can not run, or 0 when the solution is unique i.e when B.shape[1] == 1. I'll fix this ASAP, in the meantime please consider this bug as already acknowledged if anyone reports it. Regards. -- Irvin

1 0

Numpy 1.10.0 release
by Charles R Harris Oct. 6, 2015

Oct. 6, 2015

Hi All, It is my pleasure to release NumPy 1.10.0. Files may be found at Sourceforge <http://sourceforge.net/projects/numpy/files/NumPy/1.10.0/> and pypi <https://pypi.python.org/pypi?%3Aaction=pkg_edit&name=numpy>. This release is the result of 789 non-merge commits made by 160 developers over a period of a year and supports Python 2.6 - 2.7 and 3.2 - 3.5. NumPy 1.10.0 Release Notes ************************** This release supports Python 2.6 - 2.7 and 3.2 - 3.5. Highlights ========== * numpy.distutils now supports parallel compilation via the --parallel/-j argument passed to setup.py build * numpy.distutils now supports additional customization via site.cfg to control compilation parameters, i.e. runtime libraries, extra linking/compilation flags. * Addition of *np.linalg.multi_dot*: compute the dot product of two or more arrays in a single function call, while automatically selecting the fastest evaluation order. * The new function `np.stack` provides a general interface for joining a sequence of arrays along a new axis, complementing `np.concatenate` for joining along an existing axis. * Addition of `nanprod` to the set of nanfunctions. * Support for the '@' operator in Python 3.5. Dropped Support: * The _dotblas module has been removed. CBLAS Support is now in Multiarray. * The testcalcs.py file has been removed. * The polytemplate.py file has been removed. * npy_PyFile_Dup and npy_PyFile_DupClose have been removed from npy_3kcompat.h. * splitcmdline has been removed from numpy/distutils/exec_command.py. * try_run and get_output have been removed from numpy/distutils/command/config.py * The a._format attribute is no longer supported for array printing. * Keywords ``skiprows`` and ``missing`` removed from np.genfromtxt. * Keyword ``old_behavior`` removed from np.correlate. Future Changes: * In array comparisons like ``arr1 == arr2``, many corner cases involving strings or structured dtypes that used to return scalars now issue ``FutureWarning`` or ``DeprecationWarning``, and in the future will be change to either perform elementwise comparisons or raise an error. * The SafeEval class will be removed. * The alterdot and restoredot functions will be removed. See below for more details on these changes. Compatibility notes =================== numpy version string ~~~~~~~~~~~~~~~~~~~~ The numpy version string for development builds has been changed from ``x.y.z.dev-githash`` to ``x.y.z.dev0+githash`` (note the +) in order to comply with PEP 440. relaxed stride checking ~~~~~~~~~~~~~~~~~~~~~~~ NPY_RELAXED_STRIDE_CHECKING is now true by default. Concatenation of 1d arrays along any but ``axis=0`` raises ``IndexError`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Using axis != 0 has raised a DeprecationWarning since NumPy 1.7, it now raises an error. *np.ravel*, *np.diagonal* and *np.diag* now preserve subtypes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ There was inconsistent behavior between *x.ravel()* and *np.ravel(x)*, as well as between *x.diagonal()* and *np.diagonal(x)*, with the methods preserving subtypes while the functions did not. This has been fixed and the functions now behave like the methods, preserving subtypes except in the case of matrices. Matrices are special cased for backward compatibility and still return 1-D arrays as before. If you need to preserve the matrix subtype, use the methods instead of the functions. *rollaxis* and *swapaxes* always return a view ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Previously, a view was returned except when no change was made in the order of the axes, in which case the input array was returned. A view is now returned in all cases. *nonzero* now returns base ndarrays ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Previously, an inconsistency existed between 1-D inputs (returning a base ndarray) and higher dimensional ones (which preserved subclasses). Behavior has been unified, and the return will now be a base ndarray. Subclasses can still override this behavior by providing their own *nonzero* method. C API ~~~~~ The changes to *swapaxes* also apply to the *PyArray_SwapAxes* C function, which now returns a view in all cases. The changes to *nonzero* also apply to the *PyArray_Nonzero* C function, which now returns a base ndarray in all cases. The dtype structure (PyArray_Descr) has a new member at the end to cache its hash value. This shouldn't affect any well-written applications. The change to the concatenation function DeprecationWarning also affects PyArray_ConcatenateArrays, recarray field return types ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Previously the returned types for recarray fields accessed by attribute and by index were inconsistent, and fields of string type were returned as chararrays. Now, fields accessed by either attribute or indexing will return an ndarray for fields of non-structured type, and a recarray for fields of structured type. Notably, this affect recarrays containing strings with whitespace, as trailing whitespace is trimmed from chararrays but kept in ndarrays of string type. Also, the dtype.type of nested structured fields is now inherited. recarray views ~~~~~~~~~~~~~~ Viewing an ndarray as a recarray now automatically converts the dtype to np.record. See new record array documentation. Additionally, viewing a recarray with a non-structured dtype no longer converts the result's type to ndarray - the result will remain a recarray. 'out' keyword argument of ufuncs now accepts tuples of arrays ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When using the 'out' keyword argument of a ufunc, a tuple of arrays, one per ufunc output, can be provided. For ufuncs with a single output a single array is also a valid 'out' keyword argument. Previously a single array could be provided in the 'out' keyword argument, and it would be used as the first output for ufuncs with multiple outputs, is deprecated, and will result in a `DeprecationWarning` now and an error in the future. byte-array indices now raises an IndexError ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Indexing an ndarray using a byte-string in Python 3 now raises an IndexError instead of a ValueError. Masked arrays containing objects with arrays ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For such (rare) masked arrays, getting a single masked item no longer returns a corrupted masked array, but a fully masked version of the item. Median warns and returns nan when invalid values are encountered ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Similar to mean, median and percentile now emits a Runtime warning and returns `NaN` in slices where a `NaN` is present. To compute the median or percentile while ignoring invalid values use the new `nanmedian` or `nanpercentile` functions. Functions available from numpy.ma.testutils have changed ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All functions from numpy.testing were once available from numpy.ma.testutils but not all of them were redefined to work with masked arrays. Most of those functions have now been removed from numpy.ma.testutils with a small subset retained in order to preserve backward compatibility. In the long run this should help avoid mistaken use of the wrong functions, but it may cause import problems for some. New Features ============ Reading extra flags from site.cfg ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Previously customization of compilation of dependency libraries and numpy itself was only accomblishable via code changes in the distutils package. Now numpy.distutils reads in the following extra flags from each group of the *site.cfg*: * ``runtime_library_dirs/rpath``, sets runtime library directories to override ``LD_LIBRARY_PATH`` * ``extra_compile_args``, add extra flags to the compilation of sources * ``extra_link_args``, add extra flags when linking libraries This should, at least partially, complete user customization. *np.cbrt* to compute cube root for real floats ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *np.cbrt* wraps the C99 cube root function *cbrt*. Compared to *np.power(x, 1./3.)* it is well defined for negative real floats and a bit faster. numpy.distutils now allows parallel compilation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ By passing *--parallel=n* or *-j n* to *setup.py build* the compilation of extensions is now performed in *n* parallel processes. The parallelization is limited to files within one extension so projects using Cython will not profit because it builds extensions from single files. *genfromtxt* has a new ``max_rows`` argument ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A ``max_rows`` argument has been added to *genfromtxt* to limit the number of rows read in a single call. Using this functionality, it is possible to read in multiple arrays stored in a single file by making repeated calls to the function. New function *np.broadcast_to* for invoking array broadcasting ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *np.broadcast_to* manually broadcasts an array to a given shape according to numpy's broadcasting rules. The functionality is similar to broadcast_arrays, which in fact has been rewritten to use broadcast_to internally, but only a single array is necessary. New context manager *clear_and_catch_warnings* for testing warnings ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When Python emits a warning, it records that this warning has been emitted in the module that caused the warning, in a module attribute ``__warningregistry__``. Once this has happened, it is not possible to emit the warning again, unless you clear the relevant entry in ``__warningregistry__``. This makes is hard and fragile to test warnings, because if your test comes after another that has already caused the warning, you will not be able to emit the warning or test it. The context manager ``clear_and_catch_warnings`` clears warnings from the module registry on entry and resets them on exit, meaning that warnings can be re-raised. *cov* has new ``fweights`` and ``aweights`` arguments ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``fweights`` and ``aweights`` arguments add new functionality to covariance calculations by applying two types of weighting to observation vectors. An array of ``fweights`` indicates the number of repeats of each observation vector, and an array of ``aweights`` provides their relative importance or probability. Support for the '@' operator in Python 3.5+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Python 3.5 adds support for a matrix multiplication operator '@' proposed in PEP465. Preliminary support for that has been implemented, and an equivalent function ``matmul`` has also been added for testing purposes and use in earlier Python versions. The function is preliminary and the order and number of its optional arguments can be expected to change. New argument ``norm`` to fft functions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The default normalization has the direct transforms unscaled and the inverse transforms are scaled by :math:`1/n`. It is possible to obtain unitary transforms by setting the keyword argument ``norm`` to ``"ortho"`` (default is `None`) so that both direct and inverse transforms will be scaled by :math:`1/\\sqrt{n}`. Improvements ============ *np.digitize* using binary search ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *np.digitize* is now implemented in terms of *np.searchsorted*. This means that a binary search is used to bin the values, which scales much better for larger number of bins than the previous linear search. It also removes the requirement for the input array to be 1-dimensional. *np.poly* now casts integer inputs to float ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *np.poly* will now cast 1-dimensional input arrays of integer type to double precision floating point, to prevent integer overflow when computing the monic polynomial. It is still possible to obtain higher precision results by passing in an array of object type, filled e.g. with Python ints. *np.interp* can now be used with periodic functions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *np.interp* now has a new parameter *period* that supplies the period of the input data *xp*. In such case, the input data is properly normalized to the given period and one end point is added to each extremity of *xp* in order to close the previous and the next period cycles, resulting in the correct interpolation behavior. *np.pad* supports more input types for ``pad_width`` and ``constant_values`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``constant_values`` parameters now accepts NumPy arrays and float values. NumPy arrays are supported as input for ``pad_width``, and an exception is raised if its values are not of integral type. *np.argmax* and *np.argmin* now support an ``out`` argument ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``out`` parameter was added to *np.argmax* and *np.argmin* for consistency with *ndarray.argmax* and *ndarray.argmin*. The new parameter behaves exactly as it does in those methods. More system C99 complex functions detected and used ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All of the functions ``in complex.h`` are now detected. There are new fallback implementations of the following functions. * npy_ctan, * npy_cacos, npy_casin, npy_catan * npy_ccosh, npy_csinh, npy_ctanh, * npy_cacosh, npy_casinh, npy_catanh As a result of these improvements, there will be some small changes in returned values, especially for corner cases. *np.loadtxt* support for the strings produced by the ``float.hex`` method ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The strings produced by ``float.hex`` look like ``0x1.921fb54442d18p+1``, so this is not the hex used to represent unsigned integer types. *np.isclose* properly handles minimal values of integer dtypes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In order to properly handle minimal values of integer types, *np.isclose* will now cast to the float dtype during comparisons. This aligns its behavior with what was provided by *np.allclose*. *np.allclose* uses *np.isclose* internally. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *np.allclose* now uses *np.isclose* internally and inherits the ability to compare NaNs as equal by setting ``equal_nan=True``. Subclasses, such as *np.ma.MaskedArray*, are also preserved now. *np.genfromtxt* now handles large integers correctly ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *np.genfromtxt* now correctly handles integers larger than ``2**31-1`` on 32-bit systems and larger than ``2**63-1`` on 64-bit systems (it previously crashed with an ``OverflowError`` in these cases). Integers larger than ``2**63-1`` are converted to floating-point values. *np.load*, *np.save* have pickle backward compatibility flags ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The functions *np.load* and *np.save* have additional keyword arguments for controlling backward compatibility of pickled Python objects. This enables Numpy on Python 3 to load npy files containing object arrays that were generated on Python 2. MaskedArray support for more complicated base classes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Built-in assumptions that the baseclass behaved like a plain array are being removed. In particular, setting and getting elements and ranges will respect baseclass overrides of ``__setitem__`` and ``__getitem__``, and arithmetic will respect overrides of ``__add__``, ``__sub__``, etc. Changes ======= dotblas functionality moved to multiarray ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The cblas versions of dot, inner, and vdot have been integrated into the multiarray module. In particular, vdot is now a multiarray function, which it was not before. stricter check of gufunc signature compliance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Inputs to generalized universal functions are now more strictly checked against the function's signature: all core dimensions are now required to be present in input arrays; core dimensions with the same label must have the exact same size; and output core dimension's must be specified, either by a same label input core dimension or by a passed-in output array. views returned from *np.einsum* are writeable ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Views returned by *np.einsum* will now be writeable whenever the input array is writeable. *np.argmin* skips NaT values ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *np.argmin* now skips NaT values in datetime64 and timedelta64 arrays, making it consistent with *np.min*, *np.argmax* and *np.max*. Deprecations ============ Array comparisons involving strings or structured dtypes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Normally, comparison operations on arrays perform elementwise comparisons and return arrays of booleans. But in some corner cases, especially involving strings are structured dtypes, NumPy has historically returned a scalar instead. For example:: ### Current behaviour np.arange(2) == "foo" # -> False np.arange(2) < "foo" # -> True on Python 2, error on Python 3 np.ones(2, dtype="i4,i4") == np.ones(2, dtype="i4,i4,i4") # -> False Continuing work started in 1.9, in 1.10 these comparisons will now raise ``FutureWarning`` or ``DeprecationWarning``, and in the future they will be modified to behave more consistently with other comparison operations, e.g.:: ### Future behaviour np.arange(2) == "foo" # -> array([False, False]) np.arange(2) < "foo" # -> error, strings and numbers are not orderable np.ones(2, dtype="i4,i4") == np.ones(2, dtype="i4,i4,i4") # -> [False, False] SafeEval ~~~~~~~~ The SafeEval class in numpy/lib/utils.py is deprecated and will be removed in the next release. alterdot, restoredot ~~~~~~~~~~~~~~~~~~~~ The alterdot and restoredot functions no longer do anything, and are deprecated. pkgload, PackageLoader ~~~~~~~~~~~~~~~~~~~~~~ These ways of loading packages are now deprecated. bias, ddof arguments to corrcoef ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The values for the ``bias`` and ``ddof`` arguments to the ``corrcoef`` function canceled in the division implied by the correlation coefficient and so had no effect on the returned values. We now deprecate these arguments to ``corrcoef`` and the masked array version ``ma.corrcoef``. Because we are deprecating the ``bias`` argument to ``ma.corrcoef``, we also deprecate the use of the ``allow_masked`` argument as a positional argument, as its position will change with the removal of ``bias``. ``allow_masked`` will in due course become a keyword-only argument. dtype string representation changes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Since 1.6, creating a dtype object from its string representation, e.g. ``'f4'``, would issue a deprecation warning if the size did not correspond to an existing type, and default to creating a dtype of the default size for the type. Starting with this release, this will now raise a ``TypeError``. The only exception is object dtypes, where both ``'O4'`` and ``'O8'`` will still issue a deprecation warning. This platform-dependent representation will raise an error in the next release. In preparation for this upcoming change, the string representation of an object dtype, i.e. ``np.dtype(object).str``, no longer includes the item size, i.e. will return ``'|O'`` instead of ``'|O4'`` or ``'|O8'`` as before. Authors ======= This release contains work by the following people who contributed at least one patch to this release. The names are in alphabetical order by first name. Names followed by a "+" contributed a patch for the first time. Abdul Muneer+ Adam Williams+ Alan Briolat+ Alex Griffing Alex Willmer+ Alexander Belopolsky Alistair Muldal+ Allan Haldane+ Amir Sarabadani+ Andrea Bedini+ Andrew Dawson+ Andrew Nelson+ Antoine Pitrou+ Anton Ovchinnikov+ Antony Lee+ Behzad Nouri+ Bertrand+ Blake Griffith Bob Poekert+ Brian Kearns+ CJ Carey Carl Kleffner+ Chander G+ Charles Harris Chris Hogan+ Chris Kerr Chris Lamb+ Chris Laumann+ Christian Brodbeck+ Christian Brueffer Christoph Gohlke Cimarron Mittelsteadt Daniel da Silva Darsh P. Ranjan+ David Cournapeau David M Fobes+ David Powell+ Didrik Pinte+ Dimas Abreu Dutra Dmitry Zagorny+ Eric Firing Eric Hunsberger+ Eric Martin+ Eric Moore Eric O. LEBIGOT (EOL)+ Erik M. Bray Ernest N. Mamikonyan+ Fei Liu+ François Magimel+ Gabor Kovacs+ Gabriel-p+ Garrett-R+ George Castillo+ Gerrit Holl+ Gert-Ludwig Ingold+ Glen Mabey+ Graham Christensen+ Greg Thomsen+ Gregory R. Lee+ Helder Cesar+ Helder Oliveira+ Henning Dickten+ Ian Henriksen+ Jaime Fernandez James Camel+ James Salter+ Jan Schlüter+ Jarl Haggerty+ Jay Bourque Joel Nothman+ John Kirkham+ John Tyree+ Joris Van den Bossche+ Joseph Martinot-Lagarde Josh Warner (Mac) Juan Luis Cano Rodríguez Julian Taylor Kreiswolke+ Lars Buitinck Leonardo Donelli+ Lev Abalkin Lev Levitsky+ Malik Woods+ Maniteja Nandana+ Marshall Farrier+ Marten van Kerkwijk Martin Spacek Martin Thoma+ Masud Rahman+ Matt Newville+ Mattheus Ueckermann+ Matthew Brett Matthew Craig+ Michael Currie+ Michael Droettboom Michele Vallisneri+ Mortada Mehyar+ Nate Jensen+ Nathaniel J. Smith Nick Papior Andersen+ Nick Papior+ Nils Werner Oliver Eberle+ Patrick Peglar+ Paul Jacobson Pauli Virtanen Peter Iannucci+ Ralf Gommers Richard Barnes+ Ritta Narita+ Robert Johansson+ Robert LU+ Robert McGibbon+ Ryan Blakemore+ Ryan Nelson+ Sandro Tosi Saullo Giovani+ Sebastian Berg Sebastien Gouezel+ Simon Gibbons+ Simon Guillot+ Stefan Eng+ Stefan Otte+ Stefan van der Walt Stephan Hoyer+ Stuart Berg+ Sturla Molden+ Thomas A Caswell+ Thomas Robitaille Tim D. Smith+ Tom Krauss+ Tom Poole+ Toon Verstraelen+ Ulrich Seidl Valentin Haenel Vraj Mohan+ Warren Weckesser Wendell Smith+ Yaroslav Halchenko Yotam Doron Yousef Hamza+ Yuval Langer+ Yuxiang Wang+ Zbigniew Jędrzejewski-Szmek+ cel+ chebee7i+ empeeu+ endolith hannaro+ immerrr jmrosen155+ jnothman kanhua+ mbyt+ mlai+ styr+ tdihp+ wim glenn+ yolanda15+ Åsmund Hjulstad+ Enjjoy, Chuck

1 0