Mailman 3 January 2009 - SciPy-Dev

trunk open for 0.8 development
by Jarrod Millman Jan. 7, 2009

Jan. 7, 2009

I created a 0.7.x branch and the trunk is now open for 0.8 development. There a few small issues that need to be resolved before tagging the 0.7.0rc1, but those should be resolved shortly. The first release candidate will be out in a day or two with 0.7.0 to hopefully follow quickly afterward. Cheers, Jarrod

2 1

Abstract vectors in optimization
by Ben FrantzDale Jan. 7, 2009

Jan. 7, 2009

I've started playing with SciPy for numerical optimization. So far, it looks like a great set of useful algorithm implementations. If I understand the architecture correctly, there is one change that could make these libraries even more powerful, particularly for large-scale scientific computing. The optimization algorithms seem to be designed to work only with NumPy vectors. Mathematically, though, many of these algorithms can operate on an arbitrary Hilbert space, so the algoirhtms can be slightly rewritten to work with any objects that support addition, scalar multiplication, and provide an inner product, and a few other things. If these functions could be made more general in this way, I could see the SciPy optimization tools being extremely useful for large-scale optimization problems in which people already have the cost function, its gradient, and the representation of a state vector implemented (e.g., in C++). It would be easy for them to provide a Hilbert-space interface to Python. For example, the nonlinear CG solver, fmin_cg, takes a cost function, and an initial state, x0, and among other things, a number describing a norm to use to test for convergence. At present, it requires the following functions (among others) work: x0 = asarray(x0).flatten() len(x0) vecnorm(gfk, ord=norm) # where gfk is the derivative vector and norm is passed in numpy.dot(gfk,gfk) It was easy for me (new to Python, seasoned in C++) to rewrite this function to work with a class providing a Hilbert-space interface, thereby removing the dependence on any particular representation of the vector and gradient. For the sake of testing, I wrote a wrapper class that wraps a NumPy array. When rewriting fmin_cg to use that class, I don't need asarray(x0).flatten(), len(x0) is provided as __len__(self), norm(self, norm=2) is provided, and inner_product(self,other) is provided, along with the arithmatic operations. Has anyone considered this approach? It really seems like a small code change that would both simplify the optimization code and make it significantly more flexible. I've tried similar approaches for generic high-performance optimization functions in C++, but I think Python's type system and memory management makes it a better language to write this sort of solver. ―Ben

3 11

release?
by Charles سمير Doutriaux Jan. 7, 2009

Jan. 7, 2009

Hi there, I was wondering if 0.7.0 was going to be released soon? We're about to do a release ourselves and I would like to pack in the latest scipy. Thanks, C.

2 1

Is scipy.stci an orphan?
by josef.pktd＠gmail.com Jan. 6, 2009

Jan. 6, 2009

I was poking around in the source tree and was curious what scipy.stci is. But there is no basic info available http://docs.scipy.org/scipy/docs/scipy.stsci/ is empty The setup.py describes it as "image array convolution functions" Is there any relationship to ndimage? Or is this orphaned code? Josef

4 10

Stopping F2Py from generating verbose output
by Stéfan van der Walt Jan. 6, 2009

Jan. 6, 2009

Hi all, It seems that somewhere along the line f2py inserts debugging prints for scipy.lib.blas.fblas. E.g. $ python -c "from scipy.lib.blas import fblas; fblas.zaxpy([1, 2], [3, 4], n=4)" zaxpy:n=4 Traceback (most recent call last): File "<string>", line 1, in <module> fblas.error: (len(y)-offy>(n-1)*abs(incy)) failed for 1st keyword n Does anybody know how to suppress those "zaxpy:n=4" style messages, or even just where they are generated? Thanks, Stéfan

3 4

make 0.7.0b1 lintian clean
by Ondrej Certik Jan. 6, 2009

Jan. 6, 2009

Hi, I am testing the debian packaging of 0.7.0b1 and lintian says: $ lintian python-scipy_0.7.0b1-1_amd64.changes W: python-scipy: unusual-interpreter ./usr/share/pyshared/scipy/sparse/linalg/eigen/lobpcg/tests/test_lobpcg.py #!pytho W: python-scipy: executable-not-elf-or-script ./usr/share/pyshared/scipy/io/matlab/tests/gen_mat5files.m W: python-scipy: executable-not-elf-or-script ./usr/share/pyshared/scipy/io/matlab/tests/gen_mat4files.m W: python-scipy: executable-not-elf-or-script ./usr/share/pyshared/scipy/io/matlab/tests/save_matfile.m The attached patch fixes this in the scipy trunk. If you agree, please apply. You can also pull from here: http://github.com/certik/scipy/tree/ondrej I am now compiling numpy 1.2.0 necessary to run scipy, I'll report back. If I have time, I'll try to send svn di commits as well. Ondrej

2 2

scipy fails with 2.6.1 on Mac OS X
by Charles سمير Doutriaux Jan. 5, 2009

Jan. 5, 2009

Hi, I'm trying to build scipy on my mac 10.5 with python 2.6.1 I get the following error message: /usr/local/bin/gfortran -Wall -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.3-i386-2.6/build/src.macosx-10.3-i386-2.6/build/ src.macosx-10.3-i386-2.6/scipy/sparse/linalg/isolve/iterative/ _iterativemodule.o build/temp.macosx-10.3-i386-2.6/build/ src.macosx-10.3-i386-2.6/fortranobject.o build/temp.macosx-10.3- i386-2.6/build/src.macosx-10.3-i386-2.6/scipy/sparse/linalg/isolve/ iterative/STOPTEST2.o build/temp.macosx-10.3-i386-2.6/build/ src.macosx-10.3-i386-2.6/scipy/sparse/linalg/isolve/iterative/ getbreak.o build/temp.macosx-10.3-i386-2.6/build/src.macosx-10.3- i386-2.6/scipy/sparse/linalg/isolve/iterative/BiCGREVCOM.o build/ temp.macosx-10.3-i386-2.6/build/src.macosx-10.3-i386-2.6/scipy/sparse/ linalg/isolve/iterative/BiCGSTABREVCOM.o build/temp.macosx-10.3- i386-2.6/build/src.macosx-10.3-i386-2.6/scipy/sparse/linalg/isolve/ iterative/CGREVCOM.o build/temp.macosx-10.3-i386-2.6/build/ src.macosx-10.3-i386-2.6/scipy/sparse/linalg/isolve/iterative/ CGSREVCOM.o build/temp.macosx-10.3-i386-2.6/build/src.macosx-10.3- i386-2.6/scipy/sparse/linalg/isolve/iterative/GMRESREVCOM.o build/ temp.macosx-10.3-i386-2.6/build/src.macosx-10.3-i386-2.6/scipy/sparse/ linalg/isolve/iterative/QMRREVCOM.o -L/usr/local/lib/gcc/i386-apple- darwin8.11.1/4.3.0 -Lbuild/temp.macosx-10.3-i386-2.6 -lgfortran -o build/lib.macosx-10.3-i386-2.6/scipy/sparse/linalg/isolve/ _iterative.so -Wl,-framework -Wl,Accelerate ld warning: duplicate dylib /usr/local/lib/libgcc_s.1.dylib building 'scipy.sparse.linalg.dsolve._zsuperlu' extension compiling C sources C compiler: gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall - Wstrict-prototypes compile options: '-DNO_ATLAS_INFO=3 -DUSE_VENDOR_BLAS=1 -I/lgm/cdat/ 2.6.1/lib/python2.6/site-packages/numpy/core/include -I/lgm/cdat/2.6.1/ include/python2.6 -c' extra options: '-msse3' gcc: scipy/sparse/linalg/dsolve/_superluobject.c In file included from scipy/sparse/linalg/dsolve/_superluobject.h:8, from scipy/sparse/linalg/dsolve/_superluobject.c:5: scipy/sparse/linalg/dsolve/SuperLU/SRC/scomplex.h:60: error: conflicting types for '_Py_c_abs' /lgm/cdat/2.6.1/include/python2.6/complexobject.h:30: error: previous declaration of '_Py_c_abs' was here In file included from scipy/sparse/linalg/dsolve/_superluobject.h:8, from scipy/sparse/linalg/dsolve/_superluobject.c:5: scipy/sparse/linalg/dsolve/SuperLU/SRC/scomplex.h:60: error: conflicting types for '_Py_c_abs' /lgm/cdat/2.6.1/include/python2.6/complexobject.h:30: error: previous declaration of '_Py_c_abs' was here error: Command "gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DNO_ATLAS_INFO=3 -DUSE_VENDOR_BLAS=1 -I/lgm/cdat/ 2.6.1/lib/python2.6/site-packages/numpy/core/include -I/lgm/cdat/2.6.1/ include/python2.6 -c scipy/sparse/linalg/dsolve/_superluobject.c -o build/temp.macosx-10.3-i386-2.6/scipy/sparse/linalg/dsolve/ _superluobject.o -msse3" failed with exit status 1 I can provide you with the full scipy build log if you want. Thanks for your help, C.

2 2

sse related crash in numpy-1.2.1 and scipy-0.7.0b1
by Pieter Cristiaan de Groot Jan. 4, 2009

Jan. 4, 2009

Hello all, In addition to this post: http://www.nabble.com/Re:-scipy-on-old-CPU-crashes-td20787430.html I would like to report that numpy-1.2.1 and scipy-0.7.0b1 crash on there corresponding .test() scripts. It is a windows XP machine with an Athlon XP 2200 processor (does have SSE, doesn't have SSE2), using python 2.5.4. I found out the hard way that at least "leastsq" from "scipy.optimize" crashes. The windows crashreport points towards "_minpack.dll" if I remember correctly. Digging into the problem, it turned out that both numpy and scipy crash on there .test() scripts. Going back to numpy 1.2.0 and scipy-0.6.0-P3 solved the problem. I think it is a reasonable guess (but not more then that) that is related to SSE(2) instrucions ( see link above ). Unfortunately I didn't store any output of the test scripts, and since it is a critical machine I cannot experiment to much anymore with different versions. Still I hope that this report may help. Pieter p.s. I'm a relatively new user and was confused by the 'superpack' name in the download section. After getting familiar with the problems described above I think I understand what it means now. Would it be a good idea to put a small explanation what 'superpack' means on the website (just before, or in the downoad section)?

2 1

tagging 0.7.0rc1 this weekend? (was Numerical Recipes ...)
by Jarrod Millman Jan. 4, 2009

Jan. 4, 2009

The NR code review is basically complete. All the NR copyright issues have been resolved (the only thing left is to verify that the docstrings clearly explain the code's relation to NR). A big thanks to everyone who helped resolve the NR issues! Since this was the only thing blocking the release candidate, I am planning to create a 0.7.x branch and tag 0.7.0rc1 at the end of the weekend. Once I branch for 0.7.x, the trunk will be open for 0.8 development. Cheers, Jarrod

2 2

almost doubled the number of tests
by Jarrod Millman Jan. 3, 2009

Jan. 3, 2009

In preparation for the upcoming release, I was comparing the number of tests in 0.6.0 to the number on the trunk: 0.6.0: Ran 2266 tests in 203.503s Trunk: Ran 3990 tests in 392.357s I knew that a large number of test had been written over the last year, but I was very pleased to see that we nearly doubled the number. Thanks to everyone who contributed new tests over the last year. Increasing our test coverage is extremely important. Importantly, it makes it easier to refactor the code without worrying about regressions creeping in without being noticed. Hopefully, this trend will continue and we will add nearly 2000 new tests over the next year. Jarrod

2 1