Mailman 3 January 2017 - NumPy-Discussion

making np.gradient support unevenly spaced data
by alebarde＠gmail.com Jan. 11, 2017

Jan. 11, 2017

Hi all, I have implemented a proposed enhancement for the np.gradient function that allows to compute the gradient on non uniform grids. (PR: https://github.com/numpy/numpy/pull/8446) The proposed implementation has a behaviour/signature is similar to that of Matlab/Octave. As argument it can take: 1. A single scalar to specify a sample distance for all dimensions. 2. N scalars to specify a constant sample distance for each dimension. i.e. `dx`, `dy`, `dz`, ... 3. N arrays to specify the coordinates of the values along each dimension of F. The length of the array must match the size of the corresponding dimension 4. Any combination of N scalars/arrays with the meaning of 2. and 3. e.g., you can do the following: >>> f = np.array([[1, 2, 6], [3, 4, 5]], dtype=np.float) >>> dx = 2. >>> y = [1., 1.5, 3.5] >>> np.gradient(f, dx, y) [array([[ 1. , 1. , -0.5], [ 1. , 1. , -0.5]]), array([[ 2. , 2. , 2. ], [ 2. , 1.7, 0.5]])] It should not break any existing code since as of 1.12 only scalars or list of scalars are allowed. A possible alternative API could be pass arrays of sampling steps instead of the coordinates. On the one hand, this would have the advantage of having "differences" both in the scalar case and in the array case. On the other hand, if you are dealing with non uniformly-spaced data (e.g, data is mapped on a grid or it is a time-series), in most cases you already have the coordinates/timestamps. Therefore, in the case of difference as argument, you would almost always have a call np.diff before np.gradient. In the end, I would rather prefer the coordinates option since IMHO it is more handy, I don't think that would be too much "surprising" and it is what Matlab already does. Also, it could not easily lead to "silly" mistakes since the length have to match the size of the corresponding dimension. What do you think? Thanks Alessandro -- -------------------------------------------------------------------------- NOTICE: Dlgs 196/2003 this e-mail and any attachments thereto may contain confidential information and are intended for the sole use of the recipient(s) named above. If you are not the intended recipient of this message you are hereby notified that any dissemination or copying of this message is strictly prohibited. If you have received this e-mail in error, please notify the sender either by telephone or by e-mail and delete the material from any computer. Thank you. --------------------------------------------------------------------------

1 0

Fwd: Backslash operator A\b and np/sp.linalg.solve
by Ilhan Polat Jan. 10, 2017

Jan. 10, 2017

Hi everyone, I was stalking the deprecating the numpy.matrix discussion on the other thread and I wondered maybe the mailing list is a better place for the discussion about something I've been meaning to ask the dev members. I thought mailing lists are something we dumped using together with ICQ and geocities stuff but apparently not :-) Anyways, first thing is first: I have been in need of the ill-conditioned warning behavior of matlab (and possibly other software suites) for my own work. So I looked around in the numpy issues and found https://github.com/numpy/numpy/issues/3755 some time ago. Then I've learned from @rkern that there were C translations involved in the numpy source and frankly I couldn't even find the entry point of how the project is structured so I've switched to SciPy side where things are a bit more convenient. Next to teaching me more about f2py machinery, I have noticed that the linear algebra module is a bit less competitive than the usual level of scipy though it is definitely a personal opinion. So in order to get the ill-conditioning (or at least the condition number) I've wrapped up a PR using the expert routines of LAPACK (which is I think ready to merge) but still it is far from the contemporary software convenience that you generally get. https://github.com/scipy/scipy/pull/6775 The "assume_a" keyword introduced here is hopefully modular enough that should there be any need for more structures we can simply keep adding to the list without any backwards compatibility. It will be at least offering more options than what we have currently. The part that I would like to discuss requires a bit of intro so please bear with me. Let me copy/paste the part from the old PR: Around many places online, we can witness the rant about numpy/scipy not letting the users know about the conditioning for example Mike Croucher's blog <http://www.walkingrandomly.com/?p=5092> and numpy/numpy#3755 <https://github.com/numpy/numpy/issues/3755> Since we don't have any central backslash function that optimizes depending on the input data, should we create a function, let's name it with the matlab equivalent for the time being linsolve such that it automatically calls for the right solver? This way, we can avoid writing new functions for each LAPACK driver . As a reference here is a SO thread <http://stackoverflow.com/questions/18553210/how-to-implement-matlabs-mldivi…> that summarizes the linsolve <http://nl.mathworks.com/help/matlab/ref/linsolve.html> functionality. I'm sure you are aware, but just for completeness, the linear equation solvers are often built around the concept of polyalgorithm which is a fancy way of saying that the array is tested consecutively for certain structures and the checks are ordered in such a way that the simpler structure is tested the sooner. E.g. first check for diagonal matrix, then for upper/lower triangular then permuted triangular then symmetrical and so on. Here is also another example from AdvanPix http://www.advanpix.com/2016/ 10/07/architecture-of-linear-systems-solver/ Now, according to what I have coded and optimized as much as I can, a pure Python is not acceptable as an overhead during these checks. It would definitely be a noticeable slowdown if this was in place in the existing linalg.solve however I think this is certainly doable in the low-level C/FORTRAN level. CPython is certainly faster but I think only a straight C/FORTRAN implementation would cut it. Note that we only need the discovery of the structure then we can pass to the dedicated solver as is. Hence I'm not saying that we should recode the existing solve functionality. We already have the branching in place to ?GE/SY/HE/POSVX routines. ------- The second issue about the current linalg.solve function is when trying to solve for right inverse e.g. xA = b. Again with some copy/paste: The right inversion is currently a bit annoying, that is to say if we would like to compute, say, BA^{-1}, then the user has to explicitly transpose the explicitly transposed equation to avoid using an explicit inv(whose use should be discouraged anyways) x = scipy.linalg.solve(A.T, B.T).T. Since expert drivers come with a trans switch that can internally handle whether to solve the transposed or the regular equation, these routines avoid the A.T off-the-shelf. I am wondering what might be the best way to add a "r_inv" keyword such that the B.T is also handled at the FORTRAN level instead such that the user can simply write "solve(A,B, r_inv=True)". Because we don't have a backslash operation we could at least provide this much as convenience I guess. I would love to have go at it but I'm definitely not competent enough in C/FORTRAN at the production level so I was wondering whether I could get some help about this. Anyways, I hope I could make my point with a rather lengthy post. Please let me know if this is a plausible feature ilhan PS: In case gmail links won't be parsed, here are the inline links MC blog: http://www.walkingrandomly.com/?p=5092 SO thread : http://stackoverflow.com/questions/18553210/how-to- implement-matlabs-mldivide-a-k-a-the-backslash-operator linsolve/mldivide page : http://nl.mathworks.com/help/ matlab/ref/mldivide.html

5 9

ANN: Bokeh 0.12.4 Released
by Bryan Van de Ven Jan. 9, 2017

Jan. 9, 2017

Hi all, On behalf of the Bokeh team, I am pleased to announce the release of version 0.12.4 of Bokeh! Please see the announcement post at: https://bokeh.github.io/blog/2017/1/6/release-0-12-4/ which has more information as well as live demonstrations. If you are using Anaconda/miniconda, you can install it with conda: conda install -c bokeh bokeh Alternatively, you can also install it with pip: pip install bokeh Full information including details about how to use and obtain BokehJS are at: http://bokeh.pydata.org/en/0.12.4/docs/installation.html Issues, enhancement requests, and pull requests can be made on the Bokeh Github page: https://github.com/bokeh/bokeh Documentation is available at http://bokeh.pydata.org/en/0.12.4 There are over 200 total contributors to Bokeh and their time and effort help make Bokeh such an amazing project and community. Thank you again for your contributions. Finally (as always), for questions, technical assistance or if you're interested in contributing, questions can be directed to the Bokeh mailing list: bokeh(a)continuum.io or the Gitter Chat room: https://gitter.im/bokeh/bokeh Thanks, Bryan Van de Ven

1 0

Deprecating matrices.
by Charles R Harris Jan. 8, 2017

Jan. 8, 2017

Hi All, Just throwing this click bait out for discussion. Now that the `@` operator is available and things seem to be moving towards Python 3, especially in the classroom, we should consider the real possibility of deprecating the matrix type and later removing it. No doubt there are old scripts that require them, but older versions of numpy are available for those who need to run old scripts. Thoughts? Chuck

10 33

Proposed change in memmap offset attribute
by Loïc Estève Jan. 5, 2017

Jan. 5, 2017

Dear all, I have a PR at https://github.com/numpy/numpy/pull/8443 that proposes to change the value of the offset attribute of memmap objects. At the moment it is not the offset into the memmap file (as the docstring would lead you to believe) but this modulo mmap.ALLOCATIONGRANULARITY. It was deemed best to double-check on the mailing list whether anyone could think of a good reason why this is the case and/or whether anyone was using this property of the offset attribute. If you have comments about this proposed change, it is probably best if you do it on the PR in order to keep the discussion all in the same place. Cheers, Loïc

1 0

Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
by Peter Creasey Jan. 4, 2017

Jan. 4, 2017

> On Sun, Oct 9, 2016 at 12:59 PM, Stephan Hoyer <shoyer(a)gmail.com> wrote: > >> >> I agree with Sebastian and Nathaniel. I don't think we can deviating from >> the existing behavior (int ** int -> int) without breaking lots of existing >> code, and if we did, yes, we would need a new integer power function. >> >> I think it's better to preserve the existing behavior when it gives >> sensible results, and error when it doesn't. Adding another function >> float_power for the case that is currently broken seems like the right way >> to go. >> > I actually suspect that the amount of code broken by int**int->float may be relatively small (though extremely annoying for those that it happens to, and it would definitely be good to have statistics). I mean, Numpy silently transitioned to int32+uint64->float64 not so long ago which broke my code, but the world didn’t end. If the primary argument against int**int->float seems to be the difficulty of managing the transition, with int**int->Error being the seen as the required yet *very* painful intermediate step for the large fraction of the int**int users who didn’t care if it was int or float (e.g. the output is likely to be cast to float in the next step anyway), and fail loudly for those users who need int**int->int, then if you are prepared to risk a less conservative transition (i.e. we think that latter group is small enough) you could skip the error on users and just throw a warning for a couple of releases, along the lines of: WARNING int**int -> int is going to be deprecated in favour of int**int->float in Numpy 1.16. To avoid seeing this message, either use “from numpy import __future_float_power__” or explicitly set the type of one of your inputs to float, or use the new ipower(x,y) function for integer powers. Peter

5 7

Default type for functions that accumulate integers
by Charles R Harris Jan. 3, 2017

Jan. 3, 2017

Hi All, Currently functions like trace use the C long type as the default accumulator for integer types of lesser precision: dtype : dtype, optional > Determines the data-type of the returned array and of the accumulator > where the elements are summed. If dtype has the value None and `a` is > of integer type of precision less than the default integer > precision, then the default integer precision is used. Otherwise, > the precision is the same as that of `a`. > The problem with this is that the precision of long varies with the platform so that the result varies, see gh-8433 <https://github.com/numpy/numpy/issues/8433> for a complaint about this. There are two possible alternatives that seem reasonable to me: 1. Use 32 bit accumulators on 32 bit platforms and 64 bit accumulators on 64 bit platforms. 2. Always use 64 bit accumulators. Thoughts? Chuck

4 4

NumPy 1.12.0rc2 release.
by Charles R Harris Jan. 2, 2017

Jan. 2, 2017

Hi All, I'm pleased to announce the NumPy 1.12.0rc2 New Year's release. This release supports Python 2.7 and 3.4-3.6. Wheels for all supported Python versions may be downloaded from PiPY <https://pypi.python.org/pypi?%3Aaction=pkg_edit&name=numpy>, the tarball and zip files may be downloaded from Github <https://github.com/numpy/numpy/releases/tag/v1.12.0rc2>. The release notes and files hashes may also be found at Github <https://github.com/numpy/numpy/releases/tag/v1.12.0rc2> . NumPy 1.12.0rc 2 is the result of 413 pull requests submitted by 139 contributors and comprises a large number of fixes and improvements. Among the many improvements it is difficult to pick out just a few as standing above the others, but the following may be of particular interest or indicate areas likely to have future consequences. * Order of operations in ``np.einsum`` can now be optimized for large speed improvements. * New ``signature`` argument to ``np.vectorize`` for vectorizing with core dimensions. * The ``keepdims`` argument was added to many functions. * New context manager for testing warnings * Support for BLIS in numpy.distutils * Much improved support for PyPy (not yet finished) Enjoy, Chuck

1 0