Mailman 3 June 2016 - NumPy-Discussion

Re: [Numpy-discussion] Integers to integer powers, let's make a decision
by Peter Creasey June 4, 2016

June 4, 2016

> > +1 > > On Sat, Jun 4, 2016 at 10:22 AM, Charles R Harris > <charlesr.harris(a)gmail.com> wrote: >> Hi All, >> >> I've made a new post so that we can make an explicit decision. AFAICT, the >> two proposals are >> >> Integers to negative integer powers raise an error. >> Integers to integer powers always results in floats. >> >> My own sense is that 1. would be closest to current behavior and using a >> float exponential when a float is wanted is an explicit way to indicate that >> desire. OTOH, 2. would be the most convenient default for everyday numerical >> computation, but I think would more likely break current code. I am going to >> come down on the side of 1., which I don't think should cause too many >> problems if we start with a {Future, Deprecation}Warning explaining the >> workaround. >> >> Chuck >> +1 (grudgingly) My thoughts on this are: (i) Intuitive APIs are better, and power(a,b) suggests to a lot of (most?) readers that you are going to invoke a function like the C pow(double x, double y) on every element. Doing positive integer powers with the same function name suggests a correspondence that is in practice not that helpful. With a time machine I’d suggest a separate function for positive integer powers, however... (ii) I think that ship has sailed, and particularly with e.g. a**3 the numpy conventions are backed up by quite a bit of code, probably too much to change without a lot of problems. So I’d go with integer ^ negative integer is an error. Peter

1 0

Integers to integer powers
by Charles R Harris June 4, 2016

June 4, 2016

Hi All, There are currently several pull requests apropos integer arrays/scalars to integer powers and, because the area is messy and involves tradeoffs, I'd like to see some discussion here on the list before proceeding. *Scalars in 1.10* In [1]: 1 ** -1 Out[1]: 1.0 In [2]: int16(1) ** -1 Out[2]: 1 In [3]: int32(1) ** -1 Out[3]: 1 In [4]: int64(1) ** -1 Out[4]: 1.0 In [5]: 2 ** -1 Out[5]: 0.5 In [6]: int16(2) ** -1 Out[6]: 0 In [7]: int32(2) ** -1 Out[7]: 0 In [8]: int64(2) ** -1 Out[8]: 0.5 In [9]: 0 ** -1 --------------------------------------------------------------------------- ZeroDivisionError Traceback (most recent call last) <ipython-input-9-fd405d6cf4bc> in <module>() ----> 1 0 ** -1 ZeroDivisionError: 0.0 cannot be raised to a negative power In [10]: int16(0) ** -1 /home/charris/.local/bin/ipython:1: RuntimeWarning: divide by zero encountered in power #!/usr/bin/python /home/charris/.local/bin/ipython:1: RuntimeWarning: invalid value encountered in power #!/usr/bin/python Out[10]: -9223372036854775808 In [11]: int32(0) ** -1 Out[11]: -9223372036854775808 In [12]: int64(0) ** -1 /home/charris/.local/bin/ipython:1: RuntimeWarning: divide by zero encountered in long_scalars #!/usr/bin/python Out[12]: inf Proposed - for non-zero numbers the return type should be float. - for zero numbers a zero division error should be raised. *Scalar Arrays in 1.10* In [1]: array(1, dtype=int16) ** -1 Out[1]: 1 In [2]: array(1, dtype=int32) ** -1 Out[2]: 1 In [3]: array(1, dtype=int64) ** -1 Out[3]: 1 In [4]: array(2, dtype=int16) ** -1 Out[4]: 0 In [5]: array(2, dtype=int32) ** -1 Out[5]: 0 In [6]: array(2, dtype=int64) ** -1 Out[6]: 0 In [7]: array(0, dtype=int16) ** -1 /home/charris/.local/bin/ipython:1: RuntimeWarning: divide by zero encountered in power #!/usr/bin/python /home/charris/.local/bin/ipython:1: RuntimeWarning: invalid value encountered in power #!/usr/bin/python Out[7]: -9223372036854775808 In [8]: array(0, dtype=int32) ** -1 Out[8]: -9223372036854775808 In [9]: array(0, dtype=int64) ** -1 Out[9]: -9223372036854775808 In [10]: type(array(1, dtype=int64) ** -1) Out[10]: numpy.int64 In [11]: type(array(1, dtype=int32) ** -1) Out[11]: numpy.int64 In [12]: type(array(1, dtype=int16) ** -1) Out[12]: numpy.int64 Note that the return type is always int64 in all these cases. However, type is preserved in non-scalar arrays, although the value of int16 is not compatible with int32 and int64 for zero division. In [22]: array([0]*2, dtype=int16) ** -1 Out[22]: array([0, 0], dtype=int16) In [23]: array([0]*2, dtype=int32) ** -1 Out[23]: array([-2147483648, -2147483648], dtype=int32) In [24]: array([0]*2, dtype=int64) ** -1 Out[24]: array([-9223372036854775808, -9223372036854775808]) Proposed: - Raise an ZeroDivisionError for zero division, that is, in the ufunc. - Scalar arrays to return scalar arrays Thoughts? Chuck

11 30

SciPy 2016
by Hearne, Mike June 2, 2016

June 2, 2016

I am one of the co-chairs of the Birds of a Feather (BOF) committee at SciPy 2016, taking place this year from July 11-17. We are actively seeking moderators to propose BOF sessions on any number of topics. If someone on this list (a numpy dev, perhaps?) is interested in leading a numpy-focused session at SciPy this year, the submission form can be found here: http://scipy2016.scipy.org/ehome/146062/332970/ The BOF can take the form of a panel or open discussion. We have one submission already for a BOF focused on future plans for matplotlib development, so a parallel session on numpy future development is sure to be of general interest.' Thanks, and we'll see some of you in Austin next month! --Mike Hearne

1 0

Re: [Numpy-discussion] Changing FFT cache to a bounded LRU cache
by Nathaniel Smith June 1, 2016

June 1, 2016

On Jun 1, 2016 4:47 PM, "David Cournapeau" <cournape(a)gmail.com> wrote: > > > > On Tue, May 31, 2016 at 10:36 PM, Sturla Molden <sturla.molden(a)gmail.com> wrote: >> >> Joseph Martinot-Lagarde <contrebasse(a)gmail.com> wrote: >> >> > The problem with FFTW is that its license is more restrictive (GPL), and >> > because of this may not be suitable everywhere numpy.fft is. >> >> A lot of us use NumPy linked with MKL or Accelerate, both of which have >> some really nifty FFTs. And the license issue is hardly any worse than >> linking with them for BLAS and LAPACK, which we do anyway. We could extend >> numpy.fft to use MKL or Accelerate when they are available. > > > That's what we used to do in scipy, but it was a PITA to maintain. Contrary to blas/lapack, fft does not have a standard API, hence exposing a consistent API in python, including data layout involved quite a bit of work. > > It is better to expose those through 3rd party APIs. Fwiw Intel's new python distribution thing has numpy patched to use mkl for fft, and they're interested in pushing the relevant changes upstream. I have no idea how maintainable their patches are, since I haven't seen them -- this is just from taking to people here at pycon. -n

2 1

Changing FFT cache to a bounded LRU cache
by Lion Krischer June 1, 2016

June 1, 2016

Hi all, I was told to take this to the mailing list. Relevant pull request: https://github.com/numpy/numpy/pull/7686 NumPy's FFT implementation caches some form of execution plan for each encountered input data length. This is currently implemented as a simple dictionary which can grow without bounds. Calculating lots of different FFTs thus cause a memory leak from the users' perspective. We encountered a real world situation where this is an issue. The PR linked above proposes to replace the simple dictionary with an LRU (least recently used) cache. It will remove the least recently used pieces of data if it grows beyond a specified size (currently an arbitrary limit of 100 MB per cache). Thus almost all users will still benefit from the caches but their total memory size is now limited. Things to consider: * This adds quite some additional complexity but I could not find a simple way to achieve the same result. * What is a good limit on cache size? I used 100 MB because it works for my uses cases. Cheers! Lion

8 12

ANN: numexpr 2.6.0 released
by Francesc Alted June 1, 2016

June 1, 2016

========================= Announcing Numexpr 2.6.0 ========================= Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It wears multi-threaded capabilities, as well as support for Intel's MKL (Math Kernel Library), which allows an extremely fast evaluation of transcendental functions (sin, cos, tan, exp, log...) while squeezing the last drop of performance out of your multi-core processors. Look here for a some benchmarks of numexpr using MKL: https://github.com/pydata/numexpr/wiki/NumexprMKL Its only dependency is NumPy (MKL is optional), so it works well as an easy-to-deploy, easy-to-use, computational engine for projects that don't want to adopt other solutions requiring more heavy dependencies. What's new ========== This is a minor version bump because it introduces a new function. Also some minor fine tuning for recent CPUs has been done. More specifically: - Introduced a new re_evaluate() function for re-evaluating the previous executed array expression without any check. This is meant for accelerating loops that are re-evaluating the same expression repeatedly without changing anything else than the operands. If unsure, use evaluate() which is safer. - The BLOCK_SIZE1 and BLOCK_SIZE2 constants have been re-checked in order to find a value maximizing most of the benchmarks in bench/ directory. The new values (8192 and 16 respectively) give somewhat better results (~5%) overall. The CPU used for fine tuning is a relatively new Haswell processor (E3-1240 v3). In case you want to know more in detail what has changed in this version, see: https://github.com/pydata/numexpr/blob/master/RELEASE_NOTES.rst Where I can find Numexpr? ========================= The project is hosted at GitHub in: https://github.com/pydata/numexpr You can get the packages from PyPI as well (but not for RC releases): http://pypi.python.org/pypi/numexpr Share your experience ===================== Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data! -- Francesc Alted

1 0

Developer Meeting at EuroScipy?
by Sebastian Berg May 31, 2016

May 31, 2016

Hi all, since we had decided to do a regular developers meeting last year, how would EuroScipy (Aug. 23.-27., Erlangen, Germany) look as a possible place and time to have one? I believe EuroScipy would include a few people who were not able to come to SciPy last year, and it seems SciPy itself already sees some of the devs quite busy anyway. Not having checked back for room availability at the conference or anything, one option may be: August 24. (parallel to the second/last day of Tutorials) Does this seem like a good plan or do you have alternative suggestions or scheduling difficulties with this? Regards, Sebastian

2 1