Mailman 3 November 2015 - SciPy-Dev

Scipy developent environment using python3.4
by Luke Zoltan Kelley Nov. 15, 2015

Nov. 15, 2015

I've been playing around with development of scipy using an anaconda python2.7 environment, which has been working well. I created a new anaconda environment, cloned the scipy repo, and installed using `python setup.py build_ext --inplace`. In a recent PR I introduced an error with a python3 incompatibility, so I wanted to create a python3 environment in which to test as-well. I followed the same procedure, and it seemed to build fine, but when I try to import the `stats` module I get the error: $ python -c "import scipy.stats; scipy.stats.test()" Traceback (most recent call last): File "<string>", line 1, in <module> File "/Users/lzkelley/Programs/public/scipy/scipy/stats/__init__.py", line 338, in <module> from .stats import * File "/Users/lzkelley/Programs/public/scipy/scipy/stats/stats.py", line 180, in <module> import scipy.special as special File "/Users/lzkelley/Programs/public/scipy/scipy/special/__init__.py", line 627, in <module> from ._ufuncs import * ImportError: dlopen(/Users/lzkelley/Programs/public/scipy/scipy/special/_ufuncs.so, 2): Symbol not found: _PyCObject_Type Referenced from: /Users/lzkelley/Programs/public/scipy/scipy/special/_ufuncs.so Expected in: dynamic lookup Has anyone seen anything like this before? I think the problem has something to do with conflicts between the different versions of python. After trying to build the py34 version, I went back to my py27 environment and when running one of the test modules - the output errors were changed... For example, after running, python -c "import scipy.stats; scipy.stats.test()" I get a new error (from a module that I haven't touched): ====================================================================== ERROR: test_large_pseudo_determinant (test_multivariate.TestMultivariateNormal) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/lzkelley/Programs/public/scipy/scipy/stats/tests/test_multivariate.py", line 131, in test_large_pseudo_determinant assert_equal(scipy.linalg.det(cov), 0) File "/Users/lzkelley/Programs/public/scipy/scipy/linalg/basic.py", line 746, in det a_det, info = fdet(a1, overwrite_a=overwrite_a) TypeError: 'NoneType' object is not callable Regarding use of the inplace build instead of using 'runtests.py': The inplace build is great, because I can do tests with a running ipython session and simply do (e.g.) `reload(scipy.stats._binned_statistic); scipy.stats._binned_statistic.binned_statistic(...)` to see my changes immediately. Using something like `runtests.py` requires a re-build each time, which is extremely time-consuming. As a rookie, it's very difficult to make effective changes without running dozens of tests, quickly.

3 4

Numpy 1.10.2rc1
by Charles R Harris Nov. 12, 2015

Nov. 12, 2015

Hi All, I am pleased to announce the release of Numpy 1.10.2rc1. This release should fix the problems exposed in 1.10.1, which is not to say there are no remaining problems. Please test this thoroughly, exspecially if you experienced problems with 1.10.1. Julian Taylor has opened an issue relating to cblas detection on Debian (and probably Debian derived distributions) that is not dealt with in this release. Hopefully a solution will be available before the final. To all who reported issues with 1.10.1 and to those who helped close them, a big thank you. Source and binary files may be found on Sourceforge <https://sourceforge.net/projects/numpy/files/NumPy/1.10.1/>. Chuck

1 0

Random matrices
by Philip DeBoer Nov. 11, 2015

Nov. 11, 2015

I would like to expose an interface for random matrix generation on scipy, in particular, SO(N) and O(N) functionality. I wrote some code to generate random correlation matrices that could also be exposed. These seem like basic multi-dimensional results that would be useful in a wide variety of situations. There is a "random_rot" function generating SO(N) matrices in scipy/linalg/tests/test_decomp. It has a bug that was since fixed in the MDP package from whence it came, but was never fixed in scipy. A variation of this will generate O(N) matrices, and a little more work yields random correlation matrices. In Oct 2008 when random_rot was added, it was suggested that linalg was a good place for these functions. Is that still the case? Or should they be somewhere in scipy.random? Do we need a new place for random matrices? I have not contributed to scipy before. Let me know what the steps are and I will do my best to take care of this. This could benefit a wide variety of users, and it would be good to have widely used code. Thanks, Philip

3 3

deleting scipy/speed repo
by Ralf Gommers Nov. 7, 2015

Nov. 7, 2015

Hi, Does anyone object to deleting https://github.com/scipy/speed? There's not more than 200 lines of code in there and no one uses that repo anymore. The code that's in there is Python/Cython/Fortran code to solve the Laplace problem. For people interested in that, this blog post from Travis is much nicer than the repo: http://technicaldiscovery.blogspot.nl/2011/06/speeding-up-python-numpy-cyth… The only thing the blog post doesn't have is the Fortran code, but it's linked in the comments and originally came from https://github.com/certik/laplace_test Conclusion: nothing to move, just delete. Cheers, Ralf

2 2

Help wanted: implementation of 3D medial axis skeletonization
by Stefan van der Walt Nov. 6, 2015

Nov. 6, 2015

Hi all, I have been approached by a group that is interested in sponsoring the development of 3D skeletonization in scikit-image. One potential starting place would be: http://www.insight-journal.org/browse/publication/181 Is anyone interested in working on this? Please get in touch either on the scikit-image mailing list or by mailing me directly. Thanks! Stéfan

1 0

Loess license
by Hassan Kibirige Nov. 5, 2015

Nov. 5, 2015

I am resurrecting the loess code that was part of the deleted `scipy.sandbox` sub-package. It turns out that R uses the same core fortran code as it was published under a permissive licence (BSD-3 spirit) [1]. The issue is, there are patches [2] that have been submitted to R and not to the upstream source [3]. The code in R still carries the permissive upstream licence [4]. Is it okay to adopt those changes? [1] http://git.io/vlpfV [2] http://git.io/vlpvF [3] http://netlib.org/a/loess [4] http://git.io/vlxj7 Hassan

3 5

Pull Request 5318 - BFGS(Hess)
by Henry Herbol Nov. 4, 2015

Nov. 4, 2015

Hey Everyone, Back in early October I had submitted a Pull Request that's still Open. I was wondering if anyone had anymore feedback in regards to the request. It was for the addition of a backtracking BFGS algorithm to work around a case in which the optimizer only has access to the gradient of the function. Henry Herbol

1 0

Retiring wiki.scipy.org?
by Pauli Virtanen Nov. 3, 2015

Nov. 3, 2015

Hi, I've been looking through the stuff that was on wiki.scipy.org, and it appears it would be the time to retire it. cf. http://pav.iki.fi/tmp/to-be-removed/localhost_8080/AllPages.html for the actual material that is there. I have the impression that: (1) Most of the material that is useful has been moved elsewhere. The download instructions, topical software, etc. are moved to scipy.org and/or numpy/scipy documentation. The Numpy/Scipy tutorials that were there have mostly been subsumed in the Numpy/Scipy documentation, or superceded by better resources elsewhere eg. scipy-lectures.org. "Numpy for Matlab users" was just added there too. The Cookbook section was converted to IPython notebooks some years ago: https://github.com/pv/SciPy-CookBook However, several of these are not so relevant today. (2) The relevance of what's left is not so clear. The rest of the content seems fairly miscellaneous. Recipes from converting code from Numeric/numarray are probably not highly useful any more. If you know something valuable there not listed above, it would be useful to point it out. Excluding edits to front/download/etc pages that are nowadays on github/scipy/scipy.org, the wiki was edited 95 times in the last 5 years; and 2 times within the last 2 years. (3) Would having a wiki be useful? Based on the content there it's not clear if there is a need for a wiki nowadays --- people can just put stuff up on github/bitbucket/pypi and blog about it. It might be useful to have a "cookbook" collection, but a github repository with ipython notebooks, or Scipy Central, sound like better solutions for today than a wiki. Thoughts? In view of the above, I would remove links to wiki.scipy.org regardless, https://github.com/scipy/scipy.org/pull/121 -- Pauli Virtanen

3 5

binned_statistic: binnumber and array shapes
by Luke Zoltan Kelley Nov. 3, 2015

Nov. 3, 2015

The docs for `scipy.stats.binned_statistic` explain the `binnumber` returned arrays as: binnumber : 1-D ndarray of ints This assigns to each observation an integer that represents the bin in which this observation falls. Array has the same length as `values`. However, it's very difficult to understand how the returned values match this description. For example: >>> a1 = [0.1, 0.1, 0.1, 0.6] >>> a2 = [2.1, 2.6, 2.1, 2.1] >>> b1 = [0.0, 0.5, 1.0] >>> b2 = [2.0, 2.5, 3.0] >>> stats = scipy.stats.binned_statistic_2d(a1, a2, None, 'count', bins=[b1,b2]) BinnedStatistic2dResult(statistic=array([[ 2., 1.], [ 1., 0.]]), x_edge=array([ 0. , 0.5, 1. ]), y_edge=array([ 2. , 2.5, 3. ]), binnumber=array([5, 6, 5, 9])) The resulting 'statistic' array makes sense; but the 'binnumber' array is... cryptic... Before being returned, [`statistic` is reshaped and cleaned-up](https://github.com/scipy/scipy/blob/master/scipy/stats/_binned_… <https://github.com/scipy/scipy/blob/master/scipy/stats/_binned_statistic.py…>) Should the same thing be happening to `binnumber`? (Unfortunately) I created an [issue for this](https://github.com/scipy/scipy/issues/5449 <https://github.com/scipy/scipy/issues/5449>), but it seemed like this (the mailing list) was probably far more appropriate; woops. One other minor point is that the docstring for `binned_statistic_2d` says that `x` and `y` can have different lengths. I think that's a mistake; they have to be the same shape right? Luke

2 2