Mailman 3 April 2009 - SciPy-Dev

Gsoc summer student
by ross smith 27 Apr '09

27 Apr '09

Hello everyone, I've been accepted by Google summer of code this year. I'm a junior at University of Michigan Ann arbor studying physics. I've been 'programming' since high school. I began with some short programs in quickbasic. after coming to college I've picked up python and been working with it for a couple years. I've helped maintain a python codebase for a research lab here and was one of two students that worked on porting it to Py3k. My other programming projects have mostly involved additions to the codebase for labs I've worked for and an odd bugfix or patch for programs I use frequently. I'm going to be spending the first part of the summer, getting familiar with the codebase for SciPy.Signal and doing some cleanup and bug hunting. The second part will be focused on improving SciPy.signal and filling some of the gaps in functionality between SciPy.signal and matlab's signal processing toolbox. I usually idle on freenode as gaurdro and aim as gaurdro1 if anyone wants to talk. best, Ross

6 6

SVD
by Nils Wagner 27 Apr '09

27 Apr '09

Hi all, FWIW, you can find a collection of numerically singular matrices that arise in real applications or that have characteristic features of problems from practice at http://www.math.sjsu.edu/singular/matrices/ Nils

1 0

Sparse SVD in SciPy -- SVDPACK[C] made available
by Stéfan van der Walt 24 Apr '09

24 Apr '09

Hi all, Michael Berry has kindly agreed to relicense SVDPACK or SVDPACKC for use in SciPy. >From http://www.netlib.org/svdpack/: """ SVDPACK comprises four numerical (iterative) methods for computing the singular value decomposition (SVD) of large sparse matrices using double precision ANSI Fortran-77. A compatible ANSI-C version (SVDPACKC) is also available. This software package implements Lanczos and subspace iteration-based methods for determining several of the largest singular triplets (singular values and corresponding left- and right-singular vectors) for large sparse matrices. """ He asked that we let him know whether we'd like to use the Fortran or the C version. Which would best suit SciPy? I have a tendency to prefer SVDPACKC, since we can reach it easily via Cython. I'd appreciate your feedback. Regards Stéfan

4 6

Request of edit rights
by AK 20 Apr '09

20 Apr '09

Hi, I'd like to request edit rights for wiki, my username is RainyDay. thanks, -andrei

4 14

0.8.0.dev5677 test status on ubuntu 64bits.
by Xavier Gnata 17 Apr '09

17 Apr '09

Hi, On my box, scipy.test() has never been so close to a perfect score :) On ubuntu 64bits up to date, I get: 3 failures due to target accuracy problems. 1 error in test_implicit (the test in completely buggy) 1 faillure in test_yv_cephes_vs_amos ====================================================================== ERROR: test_implicit (test_odr.TestODR) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/scipy/odr/tests/test_odr.py", line 88, in test_implicit out = implicit_odr.run() File "/usr/local/lib/python2.6/dist-packages/scipy/odr/odrpack.py", line 1055, in run self.output = Output(apply(odr, args, kwds)) TypeError: y must be a sequence or integer (if model is implicit) ====================================================================== FAIL: test_random_real (test_basic.TestSingleIFFT) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/scipy/fftpack/tests/test_basic.py", line 206, in test_random_real assert_array_almost_equal (y2, x) File "/usr/local/lib/python2.6/dist-packages/numpy/testing/utils.py", line 537, in assert_array_almost_equal header='Arrays are not almost equal') File "/usr/local/lib/python2.6/dist-packages/numpy/testing/utils.py", line 395, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 0.900900900901%) x: array([ 0.90677679 +1.39698386e-08j, 0.28927034 +3.25962901e-09j, 0.13077791 +6.23986125e-08j, 0.04575246 -2.03919583e-08j, 0.15793833 +2.15254765e-08j, 0.85683167 -1.69426109e-08j,... y: array([ 0.90677708, 0.28927055, 0.13077785, 0.04575244, 0.15793809, 0.85683167, 0.8643328 , 0.23614296, 0.59372395, 0.37073961, 0.30814797, 0.89584029, 0.00581159, 0.07267515, 0.77166247,... ====================================================================== FAIL: test_yn_zeros (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/scipy/special/tests/test_basic.py", line 1602, in test_yn_zeros 488.98055964441374646], rtol=1e-19) File "/usr/local/lib/python2.6/dist-packages/scipy/special/tests/test_basic.py", line 38, in assert_tol_equal verbose=verbose, header=header) File "/usr/local/lib/python2.6/dist-packages/numpy/testing/utils.py", line 395, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=1e-19, atol=0 (mismatch 100.0%) x: array([ 450.136, 463.057, 472.807, 481.274, 488.981]) y: array([ 450.136, 463.057, 472.807, 481.274, 488.981]) ====================================================================== FAIL: test_ynp_zeros (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/scipy/special/tests/test_basic.py", line 1608, in test_ynp_zeros assert_tol_equal(yvp(443, ao), 0, atol=1e-15) File "/usr/local/lib/python2.6/dist-packages/scipy/special/tests/test_basic.py", line 38, in assert_tol_equal verbose=verbose, header=header) File "/usr/local/lib/python2.6/dist-packages/numpy/testing/utils.py", line 395, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=1e-07, atol=1e-15 (mismatch 100.0%) x: array([ 1.239e-10, -8.119e-16, 3.608e-16, 5.898e-16, 1.226e-15]) y: array(0) ====================================================================== FAIL: test_yv_cephes_vs_amos (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/scipy/special/tests/test_basic.py", line 1668, in test_yv_cephes_vs_amos self.check_cephes_vs_amos(yv, yn, rtol=1e-11, atol=1e-305) File "/usr/local/lib/python2.6/dist-packages/scipy/special/tests/test_basic.py", line 1657, in check_cephes_vs_amos assert c2.imag != 0, (v, z) AssertionError: (301, 1.0) FAILED (KNOWNFAIL=2, SKIP=17, errors=1, failures=4) SKIP=17 ? I'm not sure if it is normal or not. Xavier, who really think that the testsuite is a key element of scipy.

1 0

Scheduling 0.7.1
by David Cournapeau 17 Apr '09

17 Apr '09

Hi, I would like to release 0.7.1 relatively soon. There are a few low-hanging fruits, and here is a proposed schedule. Since no new features are added for 0.7.1, I don't think we really need a beta: - RC: 11th May (Monday) - Release: 18th May (Monday) My own interests are: - distribute a 0.7.1 binary for python 2.6 on windows and maybe mac os x. - make sure 0.7.1 is a universal build on mac os x (it is broken ATM). - fix some major segfaults/memleak in scipy.signal.lfilter, some of which were regressions compared to 0.6.0 So if you have some things (only bugs/doc enhancements, no new features), please backport them ASAP, cheers, David

3 3

0.8.0.dev5656 test_pbdv bug
by Xavier Gnata 16 Apr '09

16 Apr '09

Hi, This bug is still present in 0.8.0.dev5656: ====================================================================== FAIL: test_pbdv (test_basic.TestCephes) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/scipy/special/tests/test_basic.py", line 370, in test_pbdv assert_equal(cephes.pbdv(1,0),(0.0,0.0)) File "/usr/local/lib/python2.6/dist-packages/numpy/testing/utils.py", line 182, in assert_equal assert_equal(actual[k], desired[k], 'item=%r\n%s' % (k,err_msg), verbose) File "/usr/local/lib/python2.6/dist-packages/numpy/testing/utils.py", line 189, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: item=1 ACTUAL: 1.0 DESIRED: 0.0 A bug free testsuite is a very important item to advertise scipy in a industrial context. Could someone help me to kill this old bug? (ubuntu 64bits). Xavier

7 21

the scipy mission, include finite element solver
by Ondrej Certik 16 Apr '09

16 Apr '09

Hi, the mission of scipy from scipy.org: " SciPy is open-source software for mathematics, science, and engineering. [...] The SciPy library [...] provides many user-friendly and efficient numerical routines such as routines for numerical integration and optimization. " it has solvers for ODEs using several methods. Is having a finite element (FEM) solvers for PDEs (and some ODEs too) among the scipy's goals? My group[0] would be very interested in that. Of course, this is not something, that can happen overnight. But scipy has interface to lot's of sparse solvers, so it might have some nice pythonic interface to finite element solvers for PDEs too. If you look at the topical software, PDE section: http://scipy.org/Topical_Software#head-3df99e31c89f2e8ff60a2622805f6a304c50… there are currently 3 solvers listed, FiPy (finite volumes), SfePy (regular finite elements, Robert Cimrman, some patches from me and other people), Hermes (higher order adaptive finite elements, developed at the University of Nevada/Reno[0], where I am now). There are more good opensource FEM libraries out there, some list is here: http://hpfem.org/femcodes.html the most widely used is probably libmesh, though it's GPL and as far as I know it doesn't yet have python wrappers (I wrote some swig wrappers couple years back, but it was bad --- it's on my todo list to write good wrappers in Cython for libmesh). All FEM codes need a common functionality --- you need to load a mesh (e.g. readers and writers for all the different mesh formats out there, commercial or not), you need to define your PDE, then you need to assemble the global matrices (this is the, where all the FEM codes differ, e.g. you can have lots of different bases, etc. etc.), then depending on your problem, in most cases you end up either with linear system of equations: A * x = b or (eigenproblem): A * x = lambda * B * x where A, B are (sparse) matrices, "x" is the solution, "b" is the right hand side (vector), lambda is the eigenvalue. So the goal of the FE solver is to calculate A, B and "b". It should be general enough to allow you to discretize any integral in the weak formulation of the PDE, but in most cases, A, B or "b" is all you want. Then I solve it using solvers that are in scipy, or pysparse, or petsc4py, or slepc4py. Next one has to take "x" and feed it to the finite element solver again and then tell it to automatically adapt the mesh (there are several approaches for this, you can use some not so good error estimators, like some gradients of the solution, or you can use sophisticated errror estimators, like a reference solution, calculated on a uniformly refined mesh etc.). So you iteratively adapt, until you are satisfied. Most problems are nonlinear, so one also has to use some nonlinear solvers, e.g. I wrote some Jacobian free nonlinear solvers in scipy.optimize.nonlin, but this has to be polished and improved (and also if your problem is simple enough to allow it, you can also construct the Jacobian using FEM directly, that's the best thing). Once you have the solution, you also need to save it to many different file formats for visualization, and/or provide hooks to mayavi or matplotlib (for 2D stuff). And besides all of this, one also needs to have a common functionality like doing arithmetics with the solution (e.g. sum/substract two solutions, integrate it, etc.) , project it to different meshes, etc. etc. So do you think scipy could have some default, adaptive FEM solver, that would do the job for any PDE, and then all the common functionality above, so that one can easily hook in there any other solver, just to construct the A, B, "b" matrices/vectors and then use scipy for the rest (and optionally the solver again for the adaptivity)? This is a question where you want scipy to go in the longer term. But I think a good mission statement is to provide a viable alternative to Matlab. And for that, a good PDE solver and related tools are necessary. And if scipy had all the common functionality above, so that one can easily hook up any FEM solver in there, plus there would be some default solver in scipy itself, imho that would rock. In my group[0] (we also collaborate with Robert Cimrman) we are doing all of the above and our goal is to have such a system anyway. But it occurred to us that if this could be part of scipy, it could be a win to everybody, since I could spend more time working on scipy and all scipy users would have a good FEM solver. Plus I would then have imminent interest to release soon, release often, which imho is not bad for scipy either. As to techinical side, the 2D solver is written in C++ and builds in about 3s on 8 processors, and the python wrappers use Cython. The 3D solver takes a bit longer to build I think 17s. So it would make the scipy build a bit longer, but imho scipy should be able to solve PDEs. Together with a project like SPD: http://code.google.com/p/spdproject/ which easily builds scipy, numpy, atlas, blas, ..... on all platforms from source easily, then imho we could have a viable alternative to matlab. Ondrej P.S. Currently our FE solvers is GPL, but I think that could change to BSD if there is enough interest in that (there is in me). SPD also has to be GPL, because it's based on Sage, but if there is enough interest, I guess it's not difficult to rewrite all the buildscripts from scratch and use BSD (I myself don't mind using GPL for SPD and save myself lots of time by taking advantage of Sage). [0] http://hpfem.math.unr.edu/

12 42

Re: [SciPy-dev] glipy, OpenGL IPython terminal with numpy support
by Joe Harrington 16 Apr '09

16 Apr '09

Nicolas Rougier <Nicolas.Rougier(a)loria.fr> wrote: > I've made a Python/IPython terminal using pyglet with embedded support > for numpy arrays (1D or 2D). It uses OpenGL and is pretty fast. > > It does not aim at replacing matplotlib or mayavi mlab module but is > rahter aimed at quick experimentations or debugging. > > I've setup a launchpad glipy project at: > https://launchpad.net/glipy > > as well as a rudimentary homepage: > http://webloria.loria.fr/~rougier/glipy/ > > for those who want to test it. It is still an early beta version and may > be plenty of bugs... > > > Nicolas Nicolas, this is awesome, a perfect teaching tool! I'd like to use it in my class. Will it be stable in time for Fall classes? Some thoughts: Would it be possible to (optionally) detach the graphics from the text, so that they're in separate windows and you just see the current figure? How about using (optionally) an external display program like DS9, which has great features for interactively exploring the data in a floating 2D array (scale contrast, brightness, and displayed data range with the mouse; change colormaps; blink image sets; panel images; annotate images; etc?). This package is used widely in the astronomy community. Basically, what you're doing would automate the repetitious act of sending every line's result to DS9 manually, and provide basic capability for compositing images before display. --jh--

2 1

Patching fmin_l_bfgs_b in scipy.optimize
by Gilles Rochefort 15 Apr '09

15 Apr '09

Hi, I had some troubles getting a stable version of fmin_l_bfgs_b. I got a segmentation fault as soon as I turned a different value than -1 to iprint argument. As far as I understood, this iprint parameter is directly related to some fortran printing/writing logs. It prints iteration, norm of gradient on screen and also in a file called iterate.dat .. Even recompiling the whole stuff from lapack, atlas etc.. (using gfortran only) up to scipy doesn't solve the problem. Because I definitely gave up the idea to fix the bug by recompiling, I decided to patch the way python interacts with lbfgsb fortran procedure. So, I disabled the fortran iprint and add some iprint functionnality in pure python code directly in lbfgsb.py (scipy/optimize). Doing so, I get rid of the useless iterate.dat file too. While patching, I followed by adding some few things : - an optional stopping rule based on maximum number of iterations. I think this is more usefull than the number of function evaluations because the fortran code performs a linesearch procedure inside the lbfgsb ones. - a callback procedure which allows to inspect solution, criterion, and gradient at current iterate. Maybe my patch should be usefull for someone, so I decided to share it with the scipy community. Gilles. PS: I provide two files : lbfgsb.py -- the full python script to put in scipy/optimize scipy-optimize-lbfgsb-12162008.patch -- a patch file

3 4