Mailman 3 August 2007 - SciPy-Dev

Error in signal.cascade
by Matthieu Brucher Aug. 11, 2007

Aug. 11, 2007

Hi, I've found that in line 120 in wavelets.py, the code looks for a type "Float" in module sb (alias for numpy). I think it should be float instead, no ? As I found a lot of execution bugs in "usual" functions, are they tested ? Matthieu

2 2

NumPy-1.0.3.x
by Jarrod Millman Aug. 10, 2007

Aug. 10, 2007

Hello everyone, I made a mumpy-1.0.3.x branch from the 1.0.3 tag and tried to get everything working (see changesets 3957-3961). I added back get_path to numpy/distutils/misc_util.py, which is used by Lib/odr/setup.py in scipy 0.5.2. I also tried to clean up a few issues by doing the same thing that was done to the trunk in: http://projects.scipy.org/scipy/numpy/changeset/3845 http://projects.scipy.org/scipy/numpy/changeset/3848 I am still seeing 2 problems: 1) http://projects.scipy.org/scipy/numpy/ticket/535 2) when I run scipy.test(1,10), I get: check_cosine_weighted_infinite (scipy.integrate.tests.test_quadpack.test_quad)Illegal instruction If anyone has any ideas as to what is wrong, please let me know. Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

1 0

I am volunteering to be the release manager for NumPy 1.0.3.1 and SciPy 0.5.2
by Jarrod Millman Aug. 9, 2007

Aug. 9, 2007

I volunteer to be the release manager for NumPy 1.0.3.1 and SciPy 0.5.3. In order to actually get them both released I will obviously need some help. But given the amount of work required and the number of people who have offered to help, I believe this will be doable. Given the extensive discussion about what is needed for these releases, I am fairly confident that I know what needs to be done. I will try to be very specific about what I will do and what I will need help with. Basically, I am just rewriting the plan described by Robert Kern last month. Please let me know if you have any suggestions/comments/problems with this plan and please let me know if you can commit to helping in any way. [[NOTE: I just (on Monday) hired 2 full-time programmers to work on the neuroimaging in python (NIPY) project, so they will be able to help out with bug fixing as well as testing the pre-releases on different platforms.]] Releasing NumPy 1.0.3.1 =================== On July 24th, Robert suggested making a numpy 1.0.3.1 point release. He was concerned that there were some changes in numpy.distutils that needed to cook a little longer. So I am offering to make a 1.0.3.1 release. If Travis or one of the other core NumPy developers want to make a 1.0.4 release in the next week or so, then there won't be a need for a 1.0.3.1 release. First, I will branch from the 1.0.3 tag: svn cp http://svn.scipy.org/svn/numpy/tags/1.0.3 http://svn.scipy.org/svn/numpy/branches/1.0.3 Second, I will apply all the patches necessary to build scipy from svn, but nothing else. Then I will just follow the NumPy release instructions: http://projects.scipy.org/scipy/numpy/wiki/MakingReleases I will make the tarball and source rpm; but will need help with everything else. Things will go faster if someone else can build the Windows binaries. If not, my new programmers and I will make the binaries. Finally, one of the sourceforge admins will need upload those files once we are done. (I am happy to be made an admin and upload the files myself, if it would be more convenient.) Releasing SciPy 0.5.3 ================= I will make a 0.5.3 scipy branch: svn cp http://svn.scipy.org/svn/scipy/trunk http://svn.scipy.org/svn/scipy/branches/0.5.3 >From then on normal development will continue on the trunk, but only bug fixes will be allowed on the branch. I will ask everyone to test the branch for at least 1 week depending on whether we get any bug reports. Once we are able to get the most serious bugs fixed, I will start working with everyone to build as many binaries as possible. I will rely on David Cournapeau and Andrew Straw to provide RPMs and DEBs. Again, things will go faster if someone else can build the Windows binaries. But if not, my new programmers and I will figure out how to make the binaries for Windows. We can also make the OS X binaries especially if Robert Kern is stilling willing to help. I will also draft a release announcement and give everyone time to comment on it. I will either need to get access to the sourceforge site and the PyPi records or someone will have to update them for me. Timeline ======= If this is agreeable to everyone, I will make the NumPy branch on Friday and apply the relevant patches. Then if I can get someone else to make the Windows executables and upload the files, we should be able to have a new NumPy release before the beginning of the SciPy conference. As for the 0.5.3 SciPy branch, we can discuss this in some detail if everyone is OK with the basic plan. In general, I hope that I will be able to have a 1.0.3.1 NumPy release before August 20th. Perhaps we could even make the 0.5.3 branch by the 20th. Fortunately, as David said earlier the main issue is getting a new release of NumPy out. Resources ======== As I mentioned I just hired 2 full-time programmers to work on NIPY who will be able to help me get the binaries built and tested for the different platforms. All 3 of us will be at the SciPy conference next week. So we will hopefully be able to solve whatever problems we run into very quickly given that it will be so easy to get help. Additionally, David Cournapeau has said that he is willing to help get a new release of SciPy out. He has already been busy at work squashing bugs. Sincerely, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

1 0

Typo in docstrings
by Matthieu Brucher Aug. 8, 2007

Aug. 8, 2007

Some typos for some functions docstrings: signal.qmr : residucal instead of residual signal : misses invresz() medfilt2 instead of medfilt2d signal.wiener in the description it's not kernel_size but mysize I found another typo before, but I forgot where. Next time, I'll send it directly to the list. Matthieu

1 0

FFTW performances in scipy and numpy
by David Cournapeau Aug. 7, 2007

Aug. 7, 2007

Hi Stevens, I am one of the contributor to numpy/scipy. Let me first say I am *not* the main author of the fftw wrapping for scipy, and that I am a relatively newcommer in scipy, and do not claim a deep understanding of numpy arrays. But I have been thinking a bit on the problem since I am a big user of fft and debugged some problems in the scipy code since. - about copying killing performances: I am well aware of the problem, this was only a quick hack because the performances were abysal before this hack (the plan was computed for every fft !), and I had some difficulties to follow the code for something better. At least, it made the performances acceptable. - Because I found the code difficult to follow code, I started cleaning up the sources. The real goal is to add a better mechanism to use fftw as efficiently as possible. To improve performances: I thought about several approaches, which happen to be the ones you suggest :) - making numpy data 16 bytes aligned. This one is a bit tricky. I won't bother you with the details, but generally, numpy data may not be even "word aligned". Since some archs require some kind of alignement, there are some mechanisms to get aligned buffers from unaligned buffers in numpy API; I was thinking about an additional flag "SIMD alignement", since this could be quite useful for many optimized libraries using SIMD. But maybe this does not make sense, I have not yet thought enough about it to propose anything concrete to the main numpy developers. - I have tried FFTW_UNALIGNED + FFTW_ESTIMATE plans; unfortunately, I found that the performances were worse than using FFTW_MEASURE + copy (the copies are done into aligned buffers). I have since discover that this may be due to the totally broken architecture of my main workstation (a Pentium four): on my recent macbook (On linux, 32 bits, CoreDuo2), using no copy with FFTW_UNALIGNED is much better. - The above problem is fixable if we add a mechanisme to choose plans (ESTIMATE vs MEASURE vs ... I found that for 1d cases at least, ESTIMATE vs MEASURE is what really count performance wise). - I have also tried to get two plans in parallel for each size (one SIMD, one not SIMD), but this does not work very well, because numpy arrays are almost never 16 bytes aligned, so this does not seem to worth the effort. If you are interested in more concrete results/code, I can take a look at my test programs, make them buildable, and make them available for your comments (the tests programs do not depend on python, they are pure C code using directly the C wrapping used by scipy). cheers, David

4 8

Seg fault while running scipy.test under Red Hat EL
by Jeremy Conlin Aug. 7, 2007

Aug. 7, 2007

I have just installed numpy and scipy from the svn repository and I am having a few problems. I get a seg. fault when I run scipy.test(1,10) (see results below). And I get the following error when I try to load the stats package: $ python2.5 Python 2.5 (r25:51908, Apr 10 2007, 10:37:31) [GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import scipy.stats Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.5/site-packages/scipy/stats/__init__.py", line 7, in <module> from stats import * File "/usr/local/lib/python2.5/site-packages/scipy/stats/stats.py", line 192, in <module> import scipy.linalg as linalg File "/usr/local/lib/python2.5/site-packages/scipy/linalg/__init__.py", line 8, in <module> from basic import * File "/usr/local/lib/python2.5/site-packages/scipy/linalg/basic.py", line 313, in <module> import decomp File "/usr/local/lib/python2.5/site-packages/scipy/linalg/decomp.py", line 22, in <module> from blas import get_blas_funcs File "/usr/local/lib/python2.5/site-packages/scipy/linalg/blas.py", line 14, in <module> from scipy.linalg import fblas ImportError: /usr/local/lib/python2.5/site-packages/scipy/linalg/fblas.so: undefined symbol: srotmg_ >>> I am using RedHat Enterprise Linux with gcc and g77 version 3.4.6. Can someone help me understand what is wrong with my installation and how to fix it? Thanks, Jeremy $ python2.5 Python 2.5 (r25:51908, Apr 10 2007, 10:37:31) [GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import scipy >>> scipy.test(1,10) … … uniform fourier filter for real transforms 1 ... ok gaussian filter 1 ... ok gaussian filter 2 ... ok gaussian filter 3 ... ok gaussian filter 4 ... ok gaussian filter 5 ... ok gaussian filter 6 ... ok gaussian gradient magnitude filter 1 ... ok gaussian gradient magnitude filter 2 ... ok gaussian laplace filter 1 ... ok gaussian laplace filter 2 ... ok generation of a binary structure 1 ... ok generation of a binary structure 2 ... ok generation of a binary structure 3 ... ok generation of a binary structure 4 ... ok generic filter 1Illegal instruction $

2 1

one more scipy.optimize.line_search question
by dmitrey Aug. 6, 2007

Aug. 6, 2007

hi all, I wonder what does the parameter amax=50 mean (in optimize.line_search func)? Seems like this parameter is never used in the func, moreover, amax is defined in scipy.optimize module as a func. Also, in the middle of the func it has line (optimize.py) maxiter = 10 this one seems to be very small to me. don't you think it's better to handle the param in input args? Also, don't you think that having 2 line_search funcs is ambiguous? (I mean one in scipy.optimize, python-written, and one in scipy.optimize.linesearch, binding to minpack2) Regards, D.

2 6

New coding style (docstrings) : question
by dmitrey Aug. 6, 2007

Aug. 6, 2007

Hi all, I try to rewrite scipy.optimize docstrings (as well as openopt ones) in new docstrings standard. (it was assigned to my GSoC milestones) So please take a look at the example below - is all correct? Especially I'm interested in func handlers - are they need any type describer? (see those 1 line above and some lines below of line x0 : ndarray -- the initial guess ) Regards, D. def fmin(func, x0, args=(), xtol=1e-4, ftol=1e-4, maxiter=None, maxfun=None, full_output=0, disp=1, retall=0, callback=None): """Minimize a function using the downhill simplex algorithm. :Parameters: func -- the Python function or method to be minimized. x0 : ndarray -- the initial guess. args -- extra arguments for func. callback -- an optional user-supplied function to call after each iteration. It is called as callback(xk), where xk is the current parameter vector. :Returns: (xopt, {fopt, iter, funcalls, warnflag}) xopt : ndarray -- minimizer of function fopt : number -- value of function at minimum: fopt = func(xopt) iter : number -- number of iterations funcalls : number-- number of function calls warnflag : number -- Integer warning flag: 1 : 'Maximum number of function evaluations.' 2 : 'Maximum number of iterations.' allvecs : Python list -- a list of solutions at each iteration :OtherParameters: xtol : number -- acceptable relative error in xopt for convergence. ftol : number -- acceptable relative error in func(xopt) for convergence. maxiter : number -- the maximum number of iterations to perform. maxfun : number -- the maximum number of function evaluations. full_output : number -- non-zero if fval and warnflag outputs are desired. disp : number -- non-zero to print convergence messages. retall : number -- non-zero to return list of solutions at each iteration :SeeAlso: fmin, fmin_powell, fmin_cg, fmin_bfgs, fmin_ncg -- multivariate local optimizers leastsq -- nonlinear least squares minimizer fmin_l_bfgs_b, fmin_tnc, fmin_cobyla -- constrained multivariate optimizers anneal, brute -- global optimizers fminbound, brent, golden, bracket -- local scalar minimizers fsolve -- n-dimenstional root-finding brentq, brenth, ridder, bisect, newton -- one-dimensional root-finding fixed_point -- scalar fixed-point finder Notes ----- Uses a Nelder-Mead simplex algorithm to find the minimum of function of one or more variables. """

3 2

Need some info to solve #238: cblas vs fblas
by David Cournapeau Aug. 6, 2007

Aug. 6, 2007

Hi, While trying to look at some of the pending bugs on mac os X, I came across a quite severe problem for scipy.linalg when building with gfortran and veclib on intel. I put my findings in #238: the problem seems to be that 4 functions in veclib use the "old fortran ABI" for returned values, and this causes memory corruption when used in scipy. I am not sure how to solve this problem: I tried many different things, to observe at the end that removing a file (fblaswrap) from the sources of fblas in the setup of linalg solved the problem... What is this file used for ? Can it be removed from the fblas module ? (conditionally to having found vecLib, of course). cheers, David

2 3

GSoC weekly report
by dmitrey Aug. 5, 2007

Aug. 5, 2007

hi all, This week I tried to use Matthieu's line-search routines in openopt. I need a good line-search alg that takes into account slope angle, like scipy.optimize.line_search. Unfortunately, that one seems to be obsolete and no maintained any more, noone can (or want) answer about old_fval and old_old_fval parameters (moreover, other ones are not described as well, fortunately I understood them). So, my mailing with Matthieu continues, I hope to solve all the problems raised with connection soon. Also, I began to convert openopt documentation to new standards. There I've got some troubles too, see my post and Charles R Harris answer in scipy dev mail list. rev 3209 contains changes (ticket 285) proposed by Alan Isaac (instead of those ones proposed by the ticket author). Several hours more were elapsed for (as I g\had report) already done ticket 464. The, it turned out that in Nils Wagner numpy 1.0.4dev he has asfarray(matrix([[0.3]])) being matrix, while my numpy 1.0.1 (yesterday I have updated to 1.0.4) yields numpy.ndarray. As for me, I think latter is more correct, I don't know why it was changed in more recent numpy version. Nils have promised to submit a ticket. Regards, D.

3 3