Mailman 3 January 2016 - NumPy-Discussion

Re: [Numpy-discussion] ENH: Add the function 'expand_view'
by John Kirkham Jan. 9, 2016

Jan. 9, 2016

First, off sorry for the long turnaround on responding to these questions. Below I have tried to respond to everyone's questions and comments. I have restructured the order of the messages so that my responses are a little more structured. If anybody has more thoughts or questions, please let me know. >>>> Takes an array and tacks on arbitrary dimensions on either side, which >>> is returned as a view always. Here are the relevant features: >>>> >>>&… [View More]gt; * Creates a view of the array that has the dimensions before and after >>> tacked on to it. >>>> * Takes the before and after arguments independent of each other and >>> the current shape. >>>> * Allows for read and write access to the underlying array. >>> >>> Can you expand this with some discussion of why you want this function, >>> and why you chose these specific features? (E.g. as mentioned in the PR >>> comments already, the reason broadcast_to returns a read-only array is that >>> it was decided that this was less confusing for users, not because of any >>> technical issue.) Sometimes I find that broadcasting is insufficient or gets confused with some more complex cases. Even adding the extra dimensions using `np.newaxis`/`None` doesn't always cut it. So, being able to add the dimensions with the right shape and without copying is very nice. However, I wanted to do this in a way where this would always be a view onto the original data to avoid any copying penalty. It also came in handy when I didn't know how many dimensions I would be dealing with. The alternative would be to do something doing things like `a[..., (b.ndim-1)*(None,)]` and possibly something similar with `b`, which end up being pretty gross and unreadable as opposed to doing `expand_view(a, reps_after=b.shape[1:])`, which is quite clear. The latter also conveys to reader what those dimensions are doing. Finally in cases where `None` or `np.newaxis` work fine, it is possible for you to combine them in some operation you are doing in way which you did not intend. If broadcasting would have worked in this case, you won't get an error, but you will get the wrong answer, which you may discover later than you would like. When using `expand_view` an explicit shape can be set, which when mismatched will give you an error allowing you discover the problem as soon as you run the code as opposed after when you are trying to figure out your answer is wrong. The write option I never actually use so making it readonly is fine. I just never blocked that behavior. Perhaps it would be preferable as has been suggested. Ensuring a view is nice because this remains performant for large arrays. Using a view seems reasonable here based on the fact that we are never trying to restructure the data here. Instead, we are trying to make it appear as if its shape were different. > How is this different from using np.newaxis and broadcasting? Or am I > misunderstanding this? I hope the description above also helps clarify this a bit. To be more succinct and explicit: * This can always be used even when broadcasting may get confused. * Works cleanly and easily with variable numbers of dimensions unlike `np.newaxis`/`None`. * Using explicit shapes more tightly constrains behavior with the resulting view (helping catch bugs). * Provides the reader more information about how the other dimensions should work. >> Why is this a stride_trick? >> >> I thought this looks similar to expand_dims and could maybe be implemented >> with some extra options there. So, `expand_dims` adds a single dimension with a length of 1 where specified, but not multiple. Though I am going to infer by extra options you mean change it some way that it can add multiple axes. Though I suppose you could do this and it may even be worth pursing, it ends up being a similar idea to using `np.newaxis`/`None` a few times and so I have basically the same points to make here as I made in the last section's response. Also, I feel like using `expand_dims` with more axes specified might make it a little harder to follow than just using `np.newaxis`/`None`, but it could have its merrits. >From a performance standpoint, `expand_dims` using `reshape` to add these extra dimensions. So, it has the possibility of not returning a view, but a copy, which could take some time to build if the array is large. By using our method, we guarantee a view will always be returned; so, this allocation will never be encountered. [View Less]

2 1

ANN: second release candidate for scipy 0.17.0
by Evgeni Burovski Jan. 7, 2016

Jan. 7, 2016

Hi, I'm pleased to announce the availability of the second release candidate for Scipy 0.17.0. It's two days ahead of the original schedule: based on typical development patterns, I'd like to have two weekends and a full working week before January 17th, when this rc2 is supposed to become the final release. Please try this rc and report any issues on Github tracker or scipy-dev mailing list. Source tarballs and full release notes are available from Github Releases: https://github.com/scipy/… [View More]

2 1

Numpy 1.10.4 release.
by Charles R Harris Jan. 7, 2016

Jan. 7, 2016

Hi All, I'm pleased the release of the Numpy 1.10.4 (bugs stomped) release. This release was motivated by a reported segfault, but a few additional fixes were made: * gh-6922 BUG: numpy.recarray.sort segfaults on Windows, * gh-6937 BUG: busday_offset does the wrong thing with modifiedpreceding roll, * gh-6949 BUG: Type is lost when slicing a subclass of recarray, together with one minor enhancement * gh-6950 BUG trace is not subclass aware, np.trace(ma) != ma.trace(). The sources and … [View More]

2 2

Building master issues on Windows
by G Young Jan. 7, 2016

Jan. 7, 2016

Hello all, I've recently encountered issues building numpy off master on Windows in which setup.py complains that it can't find the Advapi library in any directories (which is an empty list). I scanned the DLL under System32 and ran /sfc scannow as Administrator, and both came up clean. Is anyone else encountering (or can reproduce) this issue, or is it just me? Any suggestions about how to resolve this problem? Thanks! Greg

2 3

Subclass awareness of outer, inner, dot, ...
by Marten van Kerkwijk Jan. 6, 2016

Jan. 6, 2016

Hi All, Having just had a look at functions where astropy's quantities are silently converted to ndarray, I came across some that, in principle, are easy to solve, yet for which, as always, there is a worry about backward compatibility. So, the question arises what to do. As a specific example, consider `np.outer` which is defined in `numeric.py` and boils down to multiply(a.ravel()[:, newaxis], b.ravel()[newaxis,:], out) Since multiply is fine with subclasses, all should be well, except … [View More]

1 0

Numpy 1.10.3 release.
by Charles R Harris Jan. 6, 2016

Jan. 6, 2016

Hi All, A significant segfault problem has been reported against Numpy 1.10.2 and I want to make a quick 1.10.3 release to get it fixed. Two questions - What exactly is the release process that has been decided on? AFAIK, I should just do a source release on Sourceforge, ping Matthew to produce wheels for Mac and wait for him to put them on pypi, and then upload the sources to pypi. No windows binaries are to be produced. - Is there anything else that needs fixing for 1.10.3? Chuck

4 8

1.10.3 release tomorrow, 1.11.x branch this month.
by Charles R Harris Jan. 5, 2016

Jan. 5, 2016

Hi All, I'm going to attempt a 1.10.3 release tomorrow to address the segfault reported in #6922 <https://github.com/numpy/numpy/issues/6922>. I'd also like to branch 1.11.x sometime this month, hopefully around Jan 18 (two weeks from now). There are some unresolved issues, in particular __numpy_ufunc__, but there is plenty of accumulated material and I'd like to get something out before the tinder buildup constitutes a fire hazard. Releases generate plenty of sparks and it would be … [View More]

4 4

Re: [Numpy-discussion] deprecate random.random_integers
by G Young Jan. 4, 2016

Jan. 4, 2016

Hello all, In light of the discussion in #6910 <https://github.com/numpy/numpy/pull/6910>, I have gone ahead and deprecated random_integers in my most recent PR here <https://github.com/numpy/numpy/pull/6910>. As this is an API change (sort of), what are people's thoughts on this deprecation? Thanks! Greg

2 1

Re: [Numpy-discussion] dtype random.rand
by G Young Jan. 3, 2016

Jan. 3, 2016

Hello, Issue #6790 <https://github.com/numpy/numpy/issues/6790> had requested the enhancement of adding the dtype argument to both random.randint and random.rand. With #6910 <https://github.com/numpy/numpy/pull/6910> merged in, that addresses the first half of the request. What do people think of adding a dtype argument to random.rand? Thanks! Greg

1 0

Proposal for a new function: np.moveaxis
by Stephan Hoyer Jan. 2, 2016

Jan. 2, 2016

I've put up a pull request implementing a new function, np.moveaxis, as an alternative to np.transpose and np.rollaxis: https://github.com/numpy/numpy/pull/6630 This functionality has been discussed (even the exact function name) several times over the years, but it never made it into a pull request. The most pressing issue is that the behavior of np.rollaxis is not intuitive to most users: https://mail.scipy.org/pipermail/numpy-discussion/2010-September/052882.html https://github.com/numpy/… [View More]

4 3