Mailman 3 October 2015 - NumPy-Discussion

isfortran compatibility in numpy 1.10.
by Charles R Harris Nov. 2, 2015

Nov. 2, 2015

Hi All, The isfortran function calls a.fnc (Fortran-Not-C), which is implemented as F_CONTIGUOUS && !C_CONTIGUOUS. Before relaxed stride checking contiguous multidimensional arrays could not be both and continguous 1-D arrays were always CONTIGUOUS, but this is not longer the case. Consequently current isfortran breaks backward compatiblity. There are two suggested solutions 1. Return `a.flags.f_contiguous`. This differs for 1-D arrays, but is most consistent with the name isfortran. 2. Return `a.flags.f_contiguous and a.ndim > 1`, which would be backward compatible. It is also possible to start with 2. but add a FutureWarning and later move to 1, which it my preferred solution. See gh-6590 <https://github.com/numpy/numpy/issues/6590> for the issue. Thoughts?

4 4

Commit rights for Jonathan J. Helmus
by Nathaniel Smith Nov. 1, 2015

Nov. 1, 2015

Hi all, Jonathan J. Helmus (@jjhelmus) has been given commit rights -- let's all welcome him aboard. -n

7 6

NumFOCUS fiscal sponsorship agreement
by Nathaniel Smith Nov. 1, 2015

Nov. 1, 2015

Hi all, Now that the governance document is in place, we need to get our legal ducks in a row by signing a fiscal sponsorship agreement with NumFOCUS. The basic idea here is that there are times when you really need some kind of corporation to represent the project -- the legal system for better or worse does not understand "a bunch of folks on a mailing list" as a legal entity capable of accepting donations, or holding funds or other assets like domain names. The obvious solution is to incorporate a company to represent the project -- but incorporating a company involves lots of super-annoying paperwork. (Like, *super* annoying.) So a standard trick is that a single non-profit corporation acts as an umbrella organization providing these services to multiple projects at once, and this is called "fiscal sponsorship". You can read more about it here: https://en.wikipedia.org/wiki/Fiscal_sponsorship NumFOCUS's standard comprehensive FSA agreement can be seen here: https://docs.google.com/document/d/11YqMX9UrgfCSgiQEUzmOFyg6Ku-vED6gMxhO6J9… and we have the option of negotiating changes if there's anything we don't like. They also have a FAQ: https://docs.google.com/document/d/1zdXp07dLvkbqBrDsw96P6mkqxnWzKJuM-1f4408… I've read through the document and didn't see anything that bothered me, except that I'm not quite sure how to make the split between the steering council and numfocus subcommittee that we have in our governance model sync up with their language about the "leadership body", and in particular the language in section 10 about simple majority votes. So I've queried them about that already. In the mean time, I'd encourage anyone with an interest to look it over and speak up if you see anything that you think should be changed before we sign. Cheers, -n -- Nathaniel J. Smith -- http://vorpus.org

4 9

Back to the Future for np.lib.split
by Charles R Harris Oct. 30, 2015

Oct. 30, 2015

Hi All, This is to note that gh-6592 <https://github.com/numpy/numpy/pull/6592> proposes to accelerate a planned future change to array_split. In Numpy < 1.9 empty arrays resulting from split always had dimension (0,),regardless of the dimensions of the other arrays in the result. In Numpy 1.9 a FutureWarning was raised saying that that would change and at some point empty arrays would have dimensions consistent with the other arrays in the result. However, there was a bug in the condition for raising that warning and empty arrays that were not the last of the returned arrays already had consistent dimensions. As a result, 1.9 was half way to the future but not quite there. That bug was fixed in 1.10.0, but some folks were already relying on the future behavior and opened a bug report on 1.10, see gh-6575 <https://github.com/numpy/numpy/issues/6575>. This left us in a sticky position. The decision has been made to revert the bug fix in 1.10 so that things remain as they were in 1.9, and move completely to consistent dimensions in 1.11. The move is a bit accelerated but seems the most graceful way out of the morass. Thoughts? Chuck

1 0

[announce] matplotlib 1.5.0 released
by Thomas Caswell Oct. 30, 2015

Oct. 30, 2015

Hey all, We are pleased to finally announce the release of matplotlib 1.5.0! It has been over a year since the last feature release and we have had over 230 people contribute to this cycle. This release of matplotlib has several major new features including - Auto-redraw using the object-oriented API in interactive mode. - Most plotting functions now support labeled data API [Jan Schulz]. - Color cycling has extended to all style properties [Ben Root]. - Four new perceptually uniform color maps, including the soon-to-be default 'viridis'. [Stefan van der Walt and Nathaniel Smith]. - More included style sheets. - Many small plotting improvements. - Proposed new framework for managing the GUI toolbar and tools. - Pixel-value on mouse over for imshow [Steven Silvester] For demos of some of these features in action see this notebook: https://gist.github.com/tacaswell/72b0d579aeb54d4fbf87 which is version of the talk I presented at scipy, pydata Seattle and pygotham this summer. There will be more in-depth demos of the new features coming. This release has a new required dependency, cycler <http://matplotlib.org/cycler> , for composing complex style cycles. In 1.5.0 we have dropped official support for python 2.6 and 3.3. The next matplotlib release will be the 2.0 default-style-only release, planned for 1-2 months from now. Tom

2 1

Linking Numpy with parallel OpenBLAS
by Daπid Oct. 29, 2015

Oct. 29, 2015

I have installed all the OpenBLAS versions availables at the Fedora repos, that include openMP and pthreads versions. But Numpy installed by pip on a virtualenv seems to only link to the serial version. Is there a way to convince it to use the parallel one? Here are my libraries: (py27)[david@SQUIDS lib64]$ ls libopenblas* libopenblas64.a libopenblaso64.so.0 libopenblasp64.so.0 libopenblas64-r0.2.14.so libopenblaso.a libopenblasp.a libopenblas64.so libopenblaso-r0.2.14.so libopenblasp-r0.2.14.so libopenblas64.so.0 libopenblaso.so libopenblasp.so libopenblas.a libopenblaso.so.0 libopenblasp.so.0 libopenblaso64.a libopenblasp64.a libopenblas-r0.2.14.so libopenblaso64-r0.2.14.so libopenblasp64-r0.2.14.so libopenblas.so libopenblaso64.so libopenblasp64.so libopenblas.so.0 And importing numpy shows that the serial is the only one open: (py27)[david@SQUIDS lib64]$ lsof libopenbl* lsof: WARNING: can't stat() tracefs file system /sys/kernel/debug/tracing Output information may be incomplete. COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME ipython 2355 david mem REG 8,2 32088056 2372346 libopenblas-r0.2.14.so This is the output of np.show_config(): lapack_opt_info: libraries = ['openblas'] library_dirs = ['/usr/lib64'] define_macros = [('HAVE_CBLAS', None)] language = c blas_opt_info: libraries = ['openblas'] library_dirs = ['/usr/lib64'] define_macros = [('HAVE_CBLAS', None)] language = c openblas_info: libraries = ['openblas'] library_dirs = ['/usr/lib64'] define_macros = [('HAVE_CBLAS', None)] language = c openblas_lapack_info: libraries = ['openblas'] library_dirs = ['/usr/lib64'] define_macros = [('HAVE_CBLAS', None)] language = c blas_mkl_info: NOT AVAILABLE Thanks, /David.

2 3

Re: [Numpy-discussion] Numpy Generalized Ufuncs: Pointer Arithmetic and Segmentation Faults (Debugging?)
by eleanore.young＠artorg.unibe.ch Oct. 26, 2015

Oct. 26, 2015

Dear Jaime, dear Travis thanks for pointing out my stride errors. This just gets me every time. After trying out Travis’ suggestion to work with numba, I feel that this works best for me. Functions are easier to generalise to different data types and I can make use of my existing Python development environment that way. Thanks again for your rapid and helpful support! Best, Eleanore

1 0

Numpy Generalized Ufuncs: Pointer Arithmetic and Segmentation Faults (Debugging?)
by eleanore.young＠artorg.unibe.ch Oct. 26, 2015

Oct. 26, 2015

Dear Numpy maintainers and developers, Thanks for providing such a great numerical library! I’m currently trying to implement the Dynamic Time Warping metric as a set of generalised numpy ufuncs, but unfortunately, I have lasting issues with pointer arithmetic and segmentation faults. Is there any way that I can use GDB or some such to debug a python/numpy extension? Furthermore: is it necessary to use pointer arithmetic to access the function arguments (as seen on http://docs.scipy.org/doc/numpy/user/c-info.ufunc-tutorial.html) or is element access (operator[]) also permissible? To break it down quickly, I need to have a fast DTW distance function dist_dtw() with two vector inputs (broadcasting should be possible), two scalar parameters and one scalar output (signature: (i), (j), (), () -> ()) usable in python for a 1-Nearest Neighbor classification algorithm. The extension also implements two functions compute_envelope() and piecewise_mean_reduction() which are used for lower-bounding based on Keogh and Ratanamahatana, 2005. The source code is available at http://pastebin.com/MunNaP7V and the prominent segmentation fault happens somewhere in the chain dist_dtw() —> meta_dtw_dist() —> slow_dtw_dist(), but I fail to pin it down. Aside from my primary questions, I wonder how to approach errors/exceptions and unit testing when developing numpy ufuncs. Are there any examples apart from the numpy manual that I could use as reference implementations of generalised numpy ufuncs? I would greatly appreciate some insight into properly developing generalised ufuncs. Best, Eleanore

3 2

ANN: Scipy 0.16.1 release
by Ralf Gommers Oct. 25, 2015

Oct. 25, 2015

Hi all, I'm happy to announce the availability of the Scipy 0.16.1 release. This is a bugfix only release; it contains no new features compared to 0.16.0. The sources and binary installers can be found at: - Source tarballs: at https://github.com/scipy/scipy/releases and on PyPi. - OS X: there are wheels on PyPi, so simply install with pip. - Windows: .exe installers can be found on https://github.com/scipy/scipy/releases Cheers, Ralf ========================== SciPy 0.16.1 Release Notes ========================== SciPy 0.16.1 is a bug-fix release with no new features compared to 0.16.0. Issues closed for 0.16.1 ------------------------ - `#5077 <https://github.com/scipy/scipy/issues/5077>`__: cKDTree not indexing properly for arrays with too many elements - `#5127 <https://github.com/scipy/scipy/issues/5127>`__: Regression in 0.16.0: solve_banded errors out in patsy test suite - `#5149 <https://github.com/scipy/scipy/issues/5149>`__: linalg tests apparently cause python to crash with numpy 1.10.0b1 - `#5154 <https://github.com/scipy/scipy/issues/5154>`__: 0.16.0 fails to build on OS X; can't find Python.h - `#5173 <https://github.com/scipy/scipy/issues/5173>`__: failing stats.histogram test with numpy 1.10 - `#5191 <https://github.com/scipy/scipy/issues/5191>`__: Scipy 0.16.x - TypeError: _asarray_validated() got an unexpected... - `#5195 <https://github.com/scipy/scipy/issues/5195>`__: tarballs missing documentation source - `#5363 <https://github.com/scipy/scipy/issues/5363>`__: FAIL: test_orthogonal.test_j_roots, test_orthogonal.test_js_roots Pull requests for 0.16.1 ------------------------ - `#5088 <https://github.com/scipy/scipy/pull/5088>`__: BUG: fix logic error in cKDTree.sparse_distance_matrix - `#5089 <https://github.com/scipy/scipy/pull/5089>`__: BUG: Don't overwrite b in lfilter's FIR path - `#5128 <https://github.com/scipy/scipy/pull/5128>`__: BUG: solve_banded failed when solving 1x1 systems - `#5155 <https://github.com/scipy/scipy/pull/5155>`__: BLD: fix missing Python include for Homebrew builds. - `#5192 <https://github.com/scipy/scipy/pull/5192>`__: BUG: backport as_inexact kwarg to _asarray_validated - `#5203 <https://github.com/scipy/scipy/pull/5203>`__: BUG: fix uninitialized use in lartg 0.16 backport - `#5204 <https://github.com/scipy/scipy/pull/5204>`__: BUG: properly return error to fortran from ode_jacobian_function... - `#5207 <https://github.com/scipy/scipy/pull/5207>`__: TST: Fix TestCtypesQuad failure on Python 3.5 for Windows - `#5352 <https://github.com/scipy/scipy/pull/5352>`__: TST: sparse: silence warnings about boolean indexing - `#5355 <https://github.com/scipy/scipy/pull/5355>`__: MAINT: backports for 0.16.1 release - `#5356 <https://github.com/scipy/scipy/pull/5356>`__: REL: update Paver file to ensure sdist contents are OK for releases. - `#5382 <https://github.com/scipy/scipy/pull/5382>`__: 0.16.x backport: MAINT: work around a possible numpy ufunc loop... - `#5393 <https://github.com/scipy/scipy/pull/5393>`__: TST:special: bump tolerance levels for test_j_roots and test_js_roots - `#5417 <https://github.co

1 0

Nansum function behavior
by Charles Rilhac Oct. 24, 2015

Oct. 24, 2015

Hello, I noticed the change regarding nan function and especially nansum function. I think this choice is a big mistake. I know that Matlab and R have made this choice but it is illogical and counterintuitive. First argument is about logic. An arithmetic operation between Nothing and Nothing cannot make a figure or an object. Nothing + Object can be an object or something else, but from nothing, it cannot ensue something else than nothing. I hope you see what I mean. Secondly, it's counterintuitive and not convenient. Because, if you want to fill the result of nanfunction you can do that easily : a = np.array([[np.nan, np.nan], [1,np.nan]]) a = np.nansum(a, axis=1) print(a) array([np.nan, 1.]) a[np.isnan(a)] = 0 Whereas, if the result is already filled with zero on NaN-full rows, you cannot replace the result of NaN-full rows by NaN easily. In the case above, you cannot because you lost information about NaN-full rows. I know it is tough to come back to a previous stage but I really think that it is wrong to absolutely fill with zeros the result of arithmetic operation containing NaN. Thank for your work guys ;-)

5 6