Mailman 3 April 2012 - SciPy-User

All of the PyData videos are now up at the Marakana site
by Fernando Perez April 17, 2012

April 17, 2012

Hi folks, A number of you expressed interest in attending the PyData workshop last month and unfortunately we had very tight space restrictions. But thanks to the team at Marakana, who pitched in and were willing to film, edit and post videos for many of the talks, you can access them all here: http://marakana.com/s/2012_pydata_workshop,1090/index.html They are in 720p so you can actually read the terminals, though I think you have to click the YouTube link to be able to change the resolution. Enjoy! f

1 0

fmin_bfgs failing on simple problem
by John Salvatier April 17, 2012

April 17, 2012

Hi all! I am having a problem with the fmin_bfgs solver that's surprising to me. Here's the toy problem I've set up: from scipy.optimize import fmin_bfgs, fmin_ncg from numpy import * import numpy as np def f(x ): if x < 0: return 1.79769313e+308 else : return x + 1./x xs = fmin_bfgs(f, array( [10.]), retall = True) The solver returns [nan] as the solution. The problem is designed to be stiff: between 0 and 1, it slopes upward to infinity but between 1 and infinity, it slopes up at a slope of 1. Left of 0 the function has a "nearly infinite" value. If bfgs encounters a value that's larger than the current value, it should try a different step size, no? Why does fmin_bfgs fail in this way? Cheers, John

2 6

How to run individual failed scipy.test() tests?
by sergio April 16, 2012

April 16, 2012

Hello all, I just manually installed python 2.7 with numpy 1.6.1, scipy0.10.1 and nose version 1.1.2 under ubuntu 10.04. I would like to know how to run failed tests of numpy.test('full') and scipy.test('full') individually, and to read a description of what exactly the test does. Thanks in advance for your help, Sergio PS. In what follows I am including the summary from each test command: >>> numpy.test('full') Running unit tests for numpy NumPy version 1.6.1 NumPy is installed in /home/myProg/Python272/Linux32b/lib/python2.7/ site- packages/numpy Python version 2.7.2 (default, Apr 14 2012, 20:45:06) [GCC 4.4.3] nose version 1.1.2 ====================================================================== FAIL: test_kind.TestKind.test_all ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/nose/ case.py", line 197, in runTest self.test(*self.arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/numpy /f2py/tests/test_kind.py", line 30, in test_all 'selectedrealkind(%s): expected %r but got %r' % (i, selected_real_kind(i), selectedrealkind(i))) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/numpy /testing/utils.py", line 34, in assert_ raise AssertionError(msg) AssertionError: selectedrealkind(16): expected 10 but got 16 ---------------------------------------------------------------------- Ran 3560 tests in 259.496s FAILED (KNOWNFAIL=3, SKIP=4, failures=1) >>> scipy.test('full') ...................... ====================================================================== ERROR: test_iv_cephes_vs_amos_mass_test (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /special/tests/test_basic.py", line 1642, in test_iv_cephes_vs_amos_mass_test c1 = special.iv(v, x) RuntimeWarning: divide by zero encountered in iv ====================================================================== ERROR: test_fdtri (test_basic.TestCephes) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /special/tests/test_basic.py", line 153, in test_fdtri cephes.fdtri(1,1,0.5) RuntimeWarning: invalid value encountered in fdtri ====================================================================== ERROR: test_continuous_extra.test_cont_extra(<scipy.stats.distributions.loggamma _gen object at 0xa560c2c>, (0.4141193182605212,), 'loggamma loc, scale test') ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/nose/ case.py", line 197, in runTest self.test(*self.arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/tests/test_continuous_extra.py", line 78, in check_loc_scale m,v = distfn.stats(*arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 1631, in stats mu = self._munp(1.0,*goodargs) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 4119, in _munp return self._mom0_sc(n,*args) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 1165, in _mom0_sc self.b, args=(m,)+args)[0] File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /integrate/quadpack.py", line 247, in quad retval = _quad(func,a,b,args,full_output,epsabs,epsrel,limit,points) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /integrate/quadpack.py", line 314, in _quad return _quadpack._qagie(func,bound,infbounds,args,full_output,epsabs,epsrel, limit) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 1162, in _mom_integ0 return x**m * self.pdf(x,*args) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 1262, in pdf place(output,cond,self._pdf(*goodargs) / scale) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 4112, in _pdf return exp(c*x-exp(x)-gamln(c)) RuntimeWarning: overflow encountered in exp ====================================================================== ERROR: test_continuous_extra.test_cont_extra(<scipy.stats.distributions.lomax_ge n object at 0xa56758c>, (1.8771398388773268,), 'lomax loc, scale test') ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/nose/ case.py", line 197, in runTest self.test(*self.arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/tests/test_continuous_extra.py", line 78, in check_loc_scale m,v = distfn.stats(*arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 1617, in stats mu, mu2, g1, g2 = self._stats(*args) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 4643, in _stats mu, mu2, g1, g2 = pareto.stats(c, loc=-1.0, moments='mvsk') File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 1615, in stats mu, mu2, g1, g2 = self._stats(*args,**{'moments':moments}) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 4594, in _stats vals = 2*(bt+1.0)*sqrt(b-2.0)/((b-3.0)*sqrt(b)) RuntimeWarning: invalid value encountered in sqrt ====================================================================== ERROR: test_discrete_basic.test_discrete_extra(<scipy.stats.distributions.hyperg eom_gen object at 0xa56d7ec>, (30, 12, 6), 'hypergeom entropy nan test') ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/nose/ case.py", line 197, in runTest self.test(*self.arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/tests/test_discrete_basic.py", line 199, in check_entropy ent = distfn.entropy(*arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 6314, in entropy place(output,cond0,self.vecentropy(*goodargs)) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/numpy /lib/function_base.py", line 1862, in __call__ theout = self.thefunc(*newargs) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 6668, in _entropy lvals = where(vals==0.0,0.0,log(vals)) RuntimeWarning: divide by zero encountered in log ====================================================================== ERROR: test_discrete_basic.test_discrete_extra(<scipy.stats.distributions.hyperg eom_gen object at 0xa56d7ec>, (21, 3, 12), 'hypergeom entropy nan test') ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/nose/ case.py", line 197, in runTest self.test(*self.arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/tests/test_discrete_basic.py", line 199, in check_entropy ent = distfn.entropy(*arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 6314, in entropy place(output,cond0,self.vecentropy(*goodargs)) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/numpy /lib/function_base.py", line 1862, in __call__ theout = self.thefunc(*newargs) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 6668, in _entropy lvals = where(vals==0.0,0.0,log(vals)) RuntimeWarning: divide by zero encountered in log ====================================================================== ERROR: test_fit (test_distributions.TestFitMethod) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/tests/test_distributions.py", line 439, in test_fit vals2 = distfunc.fit(res, optimizer='powell') File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 1874, in fit vals = optimizer(func,x0,args=(ravel(data),),disp=0) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /optimize/optimize.py", line 1621, in fmin_powell fval, x, direc1 = _linesearch_powell(func, x, direc1, tol=xtol*100) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /optimize/optimize.py", line 1491, in _linesearch_powell alpha_min, fret, iter, num = brent(myfunc, full_output=1, tol=tol) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /optimize/optimize.py", line 1312, in brent brent.optimize() File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /optimize/optimize.py", line 1213, in optimize tmp2 = (x-v)*(fx-fw) RuntimeWarning: invalid value encountered in double_scalars ====================================================================== ERROR: test_fix_fit (test_distributions.TestFitMethod) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/tests/test_distributions.py", line 460, in test_fix_fit vals2 = distfunc.fit(res,fscale=1) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /stats/distributions.py", line 1874, in fit vals = optimizer(func,x0,args=(ravel(data),),disp=0) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /optimize/optimize.py", line 301, in fmin and max(abs(fsim[0]-fsim[1:])) <= ftol): RuntimeWarning: invalid value encountered in subtract ====================================================================== FAIL: test_mio.test_mat4_3d ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/nose/ case.py", line 197, in runTest self.test(*self.arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /io/matlab/tests/test_mio.py", line 740, in test_mat4_3d stream, {'a': arr}, True, '4') File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/numpy /testing/utils.py", line 1008, in assert_raises return nose.tools.assert_raises(*args,**kwargs) AssertionError: DeprecationWarning not raised ====================================================================== FAIL: test_datatypes.test_uint64_max ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/nose/ case.py", line 197, in runTest self.test(*self.arg) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /ndimage/tests/test_datatypes.py", line 57, in test_uint64_max assert_true(x[1] > (2**63)) AssertionError: False is not true ====================================================================== FAIL: Regression test for #651: better handling of badly conditioned ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/scipy /signal/tests/test_filter_design.py", line 34, in test_bad_filter assert_raises(BadCoefficients, tf2zpk, [1e-15], [1.0, 1.0]) File "/home/srojas/myProg/Python272/Linux32b/lib/python2.7/site- packages/numpy /testing/utils.py", line 1008, in assert_raises return nose.tools.assert_raises(*args,**kwargs) AssertionError: BadCoefficients not raised ---------------------------------------------------------------------- Ran 5836 tests in 2718.233s FAILED (KNOWNFAIL=14, SKIP=42, errors=8, failures=3) <nose.result.TextTestResult run=5836 errors=8 failures=3>

2 1

[SciPy-user] Looping over one variable keeping the others constant
by surfcast23 April 15, 2012

April 15, 2012

Hi All, I would like to know how one can loop over one variable incrementing it while keeping the second fixed. With the final out put being a table of values. For example hold y fixed and increment x then increment y and re-increment x so you would get something like this x,y 1,1 2,1 3,1 1,2 2,2 3,2 thanks in advance -- View this message in context: http://old.nabble.com/Looping-over-one-variable-keeping-the-others-constant… Sent from the Scipy-User mailing list archive at Nabble.com.

5 6

Contributing to SciPy guide
by Ralf Gommers April 15, 2012

April 15, 2012

Hi all, It has been pointed out by a number of people that it's not so easy to get started with contributing to SciPy, and better documentation may help here. So I've written a guide for this. It would be great to get some feedback especially from people who've found it difficult to find this information before. And if you haven't contributed before but were thinking about doing so, perhaps this is a good opportunity to get started! Pull Request: https://github.com/scipy/scipy/pull/191 Rendered guide: https://github.com/rgommers/scipy/blob/howto-contribute/doc/HOWTO_CONTRIBUT… Cheers, Ralf

2 1

ssymm missing ?
by Nicolas Pinto April 15, 2012

April 15, 2012

Hello, I'm trying to use "ssymm" from blas but it's missing on my system: % python -c "from scipy.linalg.blas import cblas; assert hasattr(cblas, 'ssymm')" Traceback (most recent call last): File "<string>", line 1, in <module> AssertionError Other functions import fine, e.g.: % python -c "from scipy.linalg.blas import cblas; assert hasattr(cblas, 'sgemm')" % python -c "import scipy as sp; print sp.__version__" 0.10.0 Is it a problem on my side ? Thanks for your help. Regards, Nicolas

2 1

[SciPy-user] optimize fmin_cg parameter passing
by aurora1625 April 15, 2012

April 15, 2012

Hi, everyone I am new to Python and Scipy and I met a problem when trying to optimize a function. My problem is: my f function has several parameters(datatype:array), it's like f = \sum lambda_i * phi_i + \zeta *(exp(lambda_i + \nu_i)) this function is about lambda, so i get fprime = \sum phi_i + \zeta *(exp(lambda_i + \nu_i)) I want to pass in an initial lambda value, but how can I pass in other parameter, phi, zeta and nu. I guess my function is about lambda, and the other parameters should be constant and fixed, but I really don't know how to pass them in. Thank you for your help! -- View this message in context: http://old.nabble.com/optimize-fmin_cg-parameter-passing-tp33679809p3367980… Sent from the Scipy-User mailing list archive at Nabble.com.

5 4

sparse matrix construction question
by Eric Carlson April 14, 2012

April 14, 2012

Hello, I have a million row sparse matrix that can be created almost instantaneously using spdiags and csr format. I need to add one column and one row, and insert some nonzeros at various (symmetric) locations in the row and columns. I can easily accomplish this using lil format, but this increases the construction to around 20 seconds from about .05 seconds. I can probably work something out in fortran + f2py pretty easily, but ... Any python sparse matrix wizards out there with some tips? Cheers, Eric Carlson

2 2

synchronizing timestamps from different systems; unpaired linear regression
by Chris Rodgers April 13, 2012

April 13, 2012

I have what seems like a straightforward problem but it is becoming more difficult than I thought. I have two different computers recording timestamps from the same stream of events. I get lists X and Y from each computer and the question is how to figure out which entry in X corresponds to which entry in Y. Complications: 1) There are an unknown number of missing or spurious events in each list. I do not know which events in X match up to which in Y. 2) The temporal offset between the two lists is unknown, because each timer begins at a different time. 3) The clocks seem to run at slightly different speeds (~0.3% difference adds up to about 10 seconds over my 1hr recording time). I know this problem is solvable because once you find the temporal offset and clock-speed ratio, the matching timestamps agree to within 10ms. That is, there is a strong linear relationship between some unknown X->Y mapping. Basically, the problem is: given list X and list Y, and specifying a certain minimum R**2 value, what is the largest set of matched points from X and Y that satisfy this R**2 value? I have tried googling "unmatched linear regression" but this must not be the right search term. One approach that I've tried is to create an analog trace for X and Y with a Gaussian centered at each timestamp, then finding the lag that optimizes the cross-correlation between the two. This is good for finding the temporal offset but can't handle the clock-speed difference. (Also it takes a really long time because the series are 1hr of data sampled at 10Hz.) Then I can choose the closest matches between X and Y and fit them with a line, which gives me the clock-difference parameter. The problem is that there are a ton of local minima created by how I choose to match up the points in X and Y, so it gets stuck on the wrong answer. Any tips? Thanks! Chris PS: my current code and test data is here: https://github.com/cxrodgers/DiscreteAnalyze -- Chris Rodgers Graduate Student Helen Wills Neuroscience Institute University of California - Berkeley

9 18

[Reminder] Summer School "Advanced Scientific Programming in Python" in Kiel, Germany
by Tiziano Zito April 13, 2012

April 13, 2012

Advanced Scientific Programming in Python ========================================= a Summer School by the G-Node and the Institute of Experimental and Applied Physics, Christian-Albrechts-Universität zu Kiel Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists actually use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques, incorporating theoretical lectures and practical exercises tailored to the needs of a programming scientist. New skills will be tested in a real programming project: we will team up to develop an entertaining scientific computer game. We use the Python programming language for the entire course. Python works as a simple programming language for beginners, but more importantly, it also works great in scientific simulations and data analysis. We show how clean language design, ease of extensibility, and the great wealth of open source libraries for scientific computing and data visualization are driving Python to become a standard tool for the programming scientist. This school is targeted at Master or PhD students and Post-docs from all areas of science. Competence in Python or in another language such as Java, C/C++, MATLAB, or Mathematica is absolutely required. Basic knowledge of Python is assumed. Participants without any prior experience with Python should work through the proposed introductory materials before the course. Date and Location ================= September 2—7, 2012. Kiel, Germany. Preliminary Program =================== Day 0 (Sun Sept 2) — Best Programming Practices - Best Practices, Development Methodologies and the Zen of Python - Version control with git - Object-oriented programming & design patterns Day 1 (Mon Sept 3) — Software Carpentry - Test-driven development, unit testing & quality assurance - Debugging, profiling and benchmarking techniques - Best practices in data visualization - Programming in teams Day 2 (Tue Sept 4) — Scientific Tools for Python - Advanced NumPy - The Quest for Speed (intro): Interfacing to C with Cython - Advanced Python I: idioms, useful built-in data structures, generators Day 3 (Wed Sept 5) — The Quest for Speed - Writing parallel applications in Python - Programming project Day 4 (Thu Sept 6) — Efficient Memory Management - When parallelization does not help: the starving CPUs problem - Advanced Python II: decorators and context managers - Programming project Day 5 (Fri Sept 7) — Practical Software Development - Programming project - The Pelita Tournament Every evening we will have the tutors' consultation hour: Tutors will answer your questions and give suggestions for your own projects. Applications ============ You can apply on-line at http://python.g-node.org Applications must be submitted before 23:59 UTC, May 1, 2012. Notifications of acceptance will be sent by June 1, 2012. No fee is charged but participants should take care of travel, living, and accommodation expenses. Candidates will be selected on the basis of their profile. Places are limited: acceptance rate last time was around 20%. Prerequisites: You are supposed to know the basics of Python to participate in the lectures. You are encouraged to go through the introductory material available on the website. Faculty ======= - Francesc Alted, Continuum Analytics Inc., USA - Pietro Berkes, Enthought Inc., UK - Valentin Haenel, Blue Brain Project, École Polytechnique Fédérale de Lausanne, Switzerland - Zbigniew Jędrzejewski-Szmek, Faculty of Physics, University of Warsaw, Poland - Eilif Muller, Blue Brain Project, École Polytechnique Fédérale de Lausanne, Switzerland - Emanuele Olivetti, NeuroInformatics Laboratory, Fondazione Bruno Kessler and University of Trento, Italy - Rike-Benjamin Schuppner, Technologit GbR, Germany - Bartosz Teleńczuk, Unité de Neurosciences Information et Complexité, Centre National de la Recherche Scientifique, France - Stéfan van der Walt, Helen Wills Neuroscience Institute, University of California Berkeley, USA - Bastian Venthur, Berlin Institute of Technology and Bernstein Focus Neurotechnology, Germany - Niko Wilbert, TNG Technology Consulting GmbH, Germany - Tiziano Zito, Institute for Theoretical Biology, Humboldt-Universität zu Berlin, Germany Organized by Christian T. Steigies and Christian Drews of the Institute of Experimental and Applied Physics, Christian-Albrechts-Universität zu Kiel , and by Zbigniew Jędrzejewski-Szmek and Tiziano Zito for the German Neuroinformatics Node of the INCF. Website: http://python.g-node.org Contact: python-info(a)g-node.org

1 0