Mailman 3 April 2018 - NumPy-Discussion

Changing the return type of np.histogramdd
by Eric Wieser 28 Apr '18

28 Apr '18

Numpy has three histogram functions - histogram, histogram2d, and histogramdd. histogram is by far the most widely used, and in the absence of weights and normalization, returns an np.intp count for each bin. histogramdd (for which histogram2d is a wrapper) returns np.float64 in all circumstances. As a contrived comparison >>> x = np.linspace(0, 1)>>> h, e = np.histogram(x*x, bins=4); h array([25, 10, 8, 7], dtype=int64)>>> h, e = np.histogramdd((x*x,), bins=4); h array([25., 10., 8., 7.]) https://github.com/numpy/numpy/issues/7845 tracks this inconsistency. The fix is now trivial: the question is, will changing the return type break people’s code? Either we should: 1. Just change it, and hope no one is broken by it 2. Add a dtype argument: - If dtype=None, behave like np.histogram - If dtype is not specified, emit a future warning recommending to use dtype=None or dtype=float - In future, change the default to None 3. Create a new better-named function histogram_nd, which can also be created without the mistake that is https://github.com/numpy/numpy/issues/10864. Thoughts? Eric

4 8

Short-circuiting equivalent of np.any or np.all?
by Nathan Goldbaum 26 Apr '18

26 Apr '18

Hi all, I was surprised recently to discover that both np.any and np.all() do not have a way to exit early: In [1]: import numpy as np In [2]: data = np.arange(1e6) In [3]: print(data[:10]) [0. 1. 2. 3. 4. 5. 6. 7. 8. 9.] In [4]: %timeit np.any(data) 724 us +- 42.4 us per loop (mean +- std. dev. of 7 runs, 1000 loops each) In [5]: data = np.zeros(int(1e6)) In [6]: %timeit np.any(data) 732 us +- 52.9 us per loop (mean +- std. dev. of 7 runs, 1000 loops each) I don't see any discussions about this on the NumPy issue tracker but perhaps I'm missing something. I'm curious if there's a way to get a fast early-terminating search in NumPy? Perhaps there's another package I can depend on that does this? I guess I could also write a bit of cython code that does this but so far this project is pure python and I don't want to deal with the packaging headache of getting wheels built and conda-forge packages set up on all platforms. Thanks for your help! -Nathan

6 9

NumPy sprint May 24-25 at BIDS and virtual meetup tomorrow
by Matti Picus 24 Apr '18

24 Apr '18

We will take advantage of a few NumPy developers being at Berkeley to hold a two day sprint May 24-25 https://scisprints.github.io/#may-numpy-developer-sprint. Everyone is welcome, drop me a note if you are thinking of coming so we can estimate numbers. As previously announced, I am hosting a video conference open discussion Wed April 25 12:00-13:00 PDT at https://meet.google.com/bmv-fbob-ezp as part of a community building effort. If there are specific topics you wish to bring up, either contact me, add a line to the Trello card here https://trello.com/c/mTcHBmqq, or bring them up at the discussion. Thanks, Matti Picus .

3 2

NpyIter_MultiNew doesn't preserve the subclass of the array even without NPY_ITER_NO_SUBTYPE flag
by Никита Карташов 22 Apr '18

22 Apr '18

Hi everyone, @cournape and I hit a problem trying to fix this issue: https://github.com/numpy/numpy/issues/10933 <https://github.com/numpy/numpy/issues/10933> In short, the C implementation for `where` depends on the array created by NpyIter_MultiNew and it seems that there is no way to get the subclass from the iterator (even when all of the initial arrays are subclasses). Here is the gist with reproduction steps: https://gist.github.com/nkartashov/6eb56f942a18c7ee237689f9c71a3b27 <https://gist.github.com/nkartashov/6eb56f942a18c7ee237689f9c71a3b27> Could anyone help us with this problem? With regards, Nikita Kartashov

1 0

Introduction: NumPy developers at BIDS
by Stefan van der Walt 19 Apr '18

19 Apr '18

Hi everyone, I am excited to report that we have completed the hiring of two full time NumPy developers at BIDS [0]. Matti Picus has done extensive work on the PyPy project and specifically cpyext, their C compatibility layer that allows PyPy to run NumPy. In the course of this work, Matti has also been contributing to NumPy itself. He will officially start on Monday. Tyler Reddy joins us from Los Alamos National Lab for a two year sabbatical. Tyler has been working mainly on SciPy, and will start at BIDS late in June. We are very excited about this opportunity to develop NumPy further, together with the NumPy community, and look forward to making in-person introductions at SciPy2018 in July. Best regards, Stéfan [0] Berkeley Institute for Data Science at UC Berkeley

7 12

Adding a return value to np.random.shuffle
by Joseph Fox-Rabinovitz 12 Apr '18

12 Apr '18

Would it break backwards compatibility to add the input as a return value to np.random.shuffle? I doubt anyone out there is relying on the None return value. The change is trivial, and allows shuffling a new array in one line instead of two: x = np.random.shuffle(np.array(some_junk)) I've implemented the change in PR#10893. Regards, - Joe

4 5

Looking for data sets for auto analysis
by Howard Page 11 Apr '18

11 Apr '18

I am working on an automated math analyzer (sort of an statistical analyst in a box) that 1. uses artificial intelligence to recognize probable candidate mathematical models for the data 2. performs standard mathematical model fits for the best candidate models found 3. computes the standard statistical goodness of fit and statistical power statistics 4. computes and report a quantitative measure of the "falsifiability" of the mathematical model 5. generates NumPy code for verification I'm looking for data sets to analyze to test out the machine learning capabilities. We've done some interesting work so far, but there is a lot of validation we need to do. We'd love to work with anyone on the group to analyze your data and show you the output logs and generated code. Howard... 404-754-8763

1 0

PR to add an initializer kwarg to ufunc.reduce (and similar functions)
by Hameer Abbasi 09 Apr '18

09 Apr '18

Hello, everyone. I’ve submitted a PR to add a initializer kwarg to ufunc.reduce. This is useful in a few cases, e.g., it allows one to supply a “default” value for identity-less ufunc reductions, and specify an initial value for reductions such as sum (other than zero.) Please feel free to review or leave feedback, (although I think Eric and Marten have picked it apart pretty well). https://github.com/numpy/numpy/pull/10635 Thanks, Hameer Sent from Astro <https://www.helloastro.com> for Mac

5 21

ENH: Adding a count parameter to np.unpackbits
by Joseph Fox-Rabinovitz 08 Apr '18

08 Apr '18

Hi, I have added PR #10855 to allow unpackbits to unpack less than the entire set of bits. This is not a very big change, and 100% backwards compatible. It serves two purposes: 1. To make packbits and unpackbits completely invertible (and prevent things like this from being necessary: https://stackoverflow.com/a/44962805/2988730) 2. To prevent an unnecessary waste of space for large arrays that are unpacked along a single dimension. Regards, - Joe

1 0

PyData Man AHL Hackathon
by David Cournapeau 07 Apr '18

07 Apr '18

Hi there, Man AHL is organizing a hackathon for various projects around PyData, including NumPy (but also SciPy, etc.: https://www.ahl.com/hackathon). They have generously offered some funding to get some contributors to help. Not many current contributors from NumPy could make the trip, so I agreed to help. As I have not contributed in NumPy for sometimes, I think it would make more sense for me to help onboarding new contributors that may be interested, and also do some code reviews instead of contributing directly. I see that we already have > 30 bugs labeled as easy, which would be a good starting point. Is there particular areas where we should focus ? regards, David

2 1