Mailman 3 July 2016 - NumPy-Discussion

ANN: numexpr 2.6.1 released
by Francesc Alted 17 Jul '16

17 Jul '16

========================= Announcing Numexpr 2.6.1 ========================= What's new ========== This is a maintenance release that fixes a performance regression in some situations. More specifically, the BLOCK_SIZE1 constant has been set to 1024 (down from 8192). This allows for better cache utilization when there are many operands and with VML. Fixes #221. Also, support for NetBSD has been added. Thanks to Thomas Klausner. In case you want to know more in detail what has changed in this version, see: https://github.com/pydata/numexpr/blob/master/RELEASE_NOTES.rst What's Numexpr ============== Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It wears multi-threaded capabilities, as well as support for Intel's MKL (Math Kernel Library), which allows an extremely fast evaluation of transcendental functions (sin, cos, tan, exp, log...) while squeezing the last drop of performance out of your multi-core processors. Look here for a some benchmarks of numexpr using MKL: https://github.com/pydata/numexpr/wiki/NumexprMKL Its only dependency is NumPy (MKL is optional), so it works well as an easy-to-deploy, easy-to-use, computational engine for projects that don't want to adopt other solutions requiring more heavy dependencies. Where I can find Numexpr? ========================= The project is hosted at GitHub in: https://github.com/pydata/numexpr You can get the packages from PyPI as well (but not for RC releases): http://pypi.python.org/pypi/numexpr Share your experience ===================== Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data! -- Francesc Alted

1 0

100 Numpy exercises complete !
by Nicolas P. Rougier 15 Jul '16

15 Jul '16

Hi all, It's my great pleasure to announce that "100 Numpy exercises" is now complete. I've also made a notebook out of them such that you can now test them on binder. https://github.com/rougier/numpy-100 If you spot errors or have better solutions to propose, PR welcome. (I'm still fighting to fix exercise #54 that does not work as expected). Nicolas

1 0

Custom Dtype/Units discussion
by Ryan May 15 Jul '16

15 Jul '16

Greetings! I've been beating my head against a wall trying to work seamlessly with pint's unit support and arrays from numpy and xarray; these same issues seem to apply to other unit frameworks as well. Last time I dug into these problems, things like custom dtypes were raised as a more extensible solution that works within numpy (and other libraries) without needing a bunch of custom support. Anyone around SciPy this week want to get together and talk about how we can move ahead? (or acquaint me with another/better path forward?) I feel like I need to get this figured out one way or another before I can move forward in my corner of the world, and I have time I can dedicate to implementing a solution. Ryan -- Ryan May

8 20

Design feedback solicitation
by Pavlyk, Oleksandr 14 Jul '16

14 Jul '16

Hi, I am new to this list, so I will start with an introduction. My name is Oleksandr Pavlyk. I now work at Intel Corp. on the Intel Distribution for Python, and previously worked at Wolfram Research for 12 years. My latest project was to write a mirror to numpy.random, named numpy.random_intel. The module uses MKL to sample from different distributions for efficiency. It provides support for different underlying algorithms for basic pseudo-random number generation, i.e. in addition to MT19937, it also provides SFMT19937, MT2203, etc. I recently published a blog about it: https://software.intel.com/en-us/blogs/2016/06/15/faster-random-number-gene… I originally attempted to simply replace numpy.random in the Intel Distribution for Python with the new module, but due to fixed seed backwards incompatibility this results in numerous test failures in numpy, scipy, pandas and other modules. Unlike numpy.random, the new module generates a vector of random numbers at a time, which can be done faster than repeatedly generating the same number of variates one at a time. The source code for the new module is not upstreamed yet, and this email is meant to solicit early community feedback to allow for faster acceptance of the proposed changes. Thank you, Oleksandr

3 4

Re: [Numpy-discussion] 1.11.1 install fails on RHEL 7.2 (IBM Power8) with Python 2.7.12 (Nathaniel Smith response) FULL INSTALL LOG
by Brian M Belgodere 14 Jul '16

14 Jul '16

1 0

Re: [Numpy-discussion] F2PY subroutine referenced in another file
by Jonathan Stickel 10 Jul '16

10 Jul '16

Andy First compile "othermodule" with fortran (not f2py): $ gfortran -shared -o libothermodule.so othermodule.f90 Then link to it when compiling "mainmodule" with f2py: $ f2py -c -L[path to libothermodule.so] -lothermodule -m mainmodule mainmodule.f90 You can also make othermodule a static library and incorporate it into mainmodule. The syntax is a little different, but I do not have an example in front of me right now. HTH, Jonathan On 7/10/16 06:00 , numpy-discussion-request(a)scipy.org wrote: > Date: Sat, 9 Jul 2016 11:57:23 -0700 (MST) > From: asbuzza <asbuzza(a)gmail.com> > To: numpy-discussion(a)scipy.org > Subject: [Numpy-discussion] F2PY subroutine referenced in another file > Message-ID: <1468090643073-43261.post(a)n7.nabble.com> > Content-Type: text/plain; charset=us-ascii > > Hello everyone, > > I am relatively new to f2py and have been trying to create a module of a > main fortran file that references a subroutine in another fortran file. I > have been doing: > >> >f2py -c -m mainmodule mainmodule.f90 othermodule.f90 > Any help would be greatly appreciated. > > Thanks, > Andy

1 0

F2PY subroutine referenced in another file
by asbuzza 09 Jul '16

09 Jul '16

Hello everyone, I am relatively new to f2py and have been trying to create a module of a main fortran file that references a subroutine in another fortran file. I have been doing: >f2py -c -m mainmodule mainmodule.f90 othermodule.f90 Any help would be greatly appreciated. Thanks, Andy -- View this message in context: http://numpy-discussion.10968.n7.nabble.com/F2PY-subroutine-referenced-in-a… Sent from the Numpy-discussion mailing list archive at Nabble.com.

1 0

scipy 0.18.0 release candidate 2
by Evgeni Burovski 08 Jul '16

08 Jul '16

On behalf of the scipy development team, I am pleased to announce the second release candidate for scipy 0.18.0. Please test it --- both the release itself on your machines and your code against this release --- and report results on the issue tracker or scipy-dev mailing list. Source tarballs and release notes are available from Github releases, https://github.com/scipy/scipy/releases/tag/v0.18.0rc2 OS X and Linux wheels are being built now, and I will upload them to PyPI when they are ready. I would like to flag two areas where additional scrutiny and testing would be most appreciated: * Precision loss in some corner cases was reported to occur on Fedora 23 and 24 with OpenBLAS, see https://github.com/scipy/scipy/issues/6286. * On Windows, a CRT incompatibility was reported when building with MSVC and MKL, see https://github.com/scipy/scipy/issues/6336. There are additional issues on s390x (https://github.com/scipy/scipy/issues/6338), so if you are using this platform, we appreciate your help resolving them. The (revised) release schedule is: 25 July: final release Thanks to everyone who contributed! Cheers, Evgeni

1 0

Remove FCompiler flags?
by Rob Nagler 07 Jul '16

07 Jul '16

I would like to remove "-fno-second-underscore" and "-Wall" from the fcompiler. I had hacked a build_clib subclass to remove the flags dynamically, but that is fragile (and is now broken). Is there a right way to remove flags? Thanks, Rob

2 2

Re: [Numpy-discussion] Added atleast_nd, request for clarification/cleanup of atleast_3d
by Nathaniel Smith 06 Jul '16

06 Jul '16

On Jul 5, 2016 11:21 PM, "Ralf Gommers" <ralf.gommers(a)gmail.com> wrote: > > > > On Wed, Jul 6, 2016 at 7:06 AM, Nathaniel Smith <njs(a)pobox.com> wrote: > >> On Jul 5, 2016 9:09 PM, "Joseph Fox-Rabinovitz" <jfoxrabinovitz(a)gmail.com> wrote: >> > >> > Hi, >> > >> > I have generalized np.atleast_1d, np.atleast_2d, np.atleast_3d with a >> > function np.atleast_nd in PR#7804 >> > (https://github.com/numpy/numpy/pull/7804). >> > >> > As a result of this PR, I have a couple of questions about >> > `np.atleast_3d`. `np.atleast_3d` appears to do something weird with >> > the dimensions: If the input is 1D, it prepends and appends a size-1 >> > dimension. If the input is 2D, it appends a size-1 dimension. This is >> > inconsistent with `np.atleast_2d`, which always prepends (as does >> > `np.atleast_nd`). >> > >> > - Is there any reason for this behavior? >> > - Can it be cleaned up (e.g., by reimplementing `np.atleast_3d` in >> > terms of `np.atleast_nd`, which is actually much simpler)? This would >> > be a slight API change since the output would not be exactly the same. >> >> Changing atleast_3d seems likely to break a bunch of stuff... >> >> Beyond that, I find it hard to have an opinion about the best design for these functions, because I don't think I've ever encountered a situation where they were actually what I wanted. I'm not a big fan of coercing dimensions in the first place, for the usual "refuse to guess" reasons. And then generally if I do want to coerce an array to another dimension, then I have some opinion about where the new dimensions should go, and/or I have some opinion about the minimum acceptable starting dimension, and/or I have a maximum dimension in mind. (E.g. "coerce 1d inputs into a column matrix; 0d or 3d inputs are an error" -- atleast_2d is zero-for-three on that requirements list.) >> >> I don't know how typical I am in this. But it does make me wonder if the atleast_* functions act as an attractive nuisance, where new users take their presence as an implicit recommendation that they are actually a useful thing to reach for, even though they... aren't that. And maybe we should be recommending folk move away from them rather than trying to extend them further? >> >> Or maybe they're totally useful and I'm just missing it. What's your use case that motivates atleast_nd? > > I think you're just missing it:) atleast_1d/2d are used quite a bit in Scipy and Statsmodels (those are the only ones I checked), and in the large majority of cases it's the best thing to use there. There's a bunch of atleast_2d calls with a transpose appended because the input needs to be treated as columns instead of rows, but that's still efficient and readable enough. I know people *use* it :-). What I'm confused about is in what situations you would invent it if it didn't exist. Can you point me to an example or two where it's "the best thing"? I actually had statsmodels in mind with my example of wanting the semantics "coerce 1d inputs into a column matrix; 0d or 3d inputs are an error". I'm surprised if there are places where you really want 0d arrays converted into 1x1, or want to allow high dimensional arrays to pass through - and if you do want to allow high dimensional arrays to pass through, then transposing might help with 2d cases but will silently mangle high-d cases, right? -n

3 3