Mailman 3 March 2009 - SciPy-User

OS-X Universal binary?
by Christopher Barker 28 Mar '09

28 Mar '09

Hi folks, I just found on the sourceforge site what is labeled as a Universal binary package -- nice! However, at least part of it is actually Intel only -- darn: >>> import scipy >>> import scipy.interpolate Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/interpolate/__init__.py", line 7, in <module> from interpolate import * File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/interpolate/interpolate.py", line 13, in <module> import scipy.special as spec File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/special/__init__.py", line 8, in <module> from basic import * File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/special/basic.py", line 8, in <module> from _cephes import * ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/special/_cephes.so, 2): no suitable image found. Did find: /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/special/_cephes.so: mach-o, but wrong architecture indeed: $ file /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/special/_cephes.so /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/special/_cephes.so: Mach-O bundle i386 Is this supposed to be Universal?? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

7 19

SciPy 2009 Conference will be Aug. 18-23
by Jarrod Millman 27 Mar '09

27 Mar '09

The subject says it all. Over the next few days, we will be updating the conference website with additional information. So if you are interested, please keep an eye on: http://conference.scipy.org/ Jarrod

1 0

SciPy / NumPy on MacOS 10.5 / Python 2.6 - 64bit
by Höfling Martin 27 Mar '09

27 Mar '09

Hey Folks, I tried to compile 1.3.0b1 with a python 2.6, compiled as 64bit version (installed to $HOME/local). Is this in general working on MacOS Leopard? I know that Michael made some attempts... For some reason - the compilation always complains about an unset MATHLIB variable although I set it to "m" here're the FLAGS I used for compilation CFLAGS= -I/Users/martin/local/include -m64 CPPFLAGS= -I/Users/martin/local/include -m64 FFLAGS= -I/Users/martin/local/include -m64 CXXFLAGS= -I/Users/martin/local/include -m64 FCFLAGS= -I/Users/martin/local/include -m64 LDFLAGS= -L/Users/martin/local/lib -m64 -arch x86_64 Best Martin Find attached the logfile from compilation process.

2 2

statistical tests notes
by josef.pktd＠gmail.com 27 Mar '09

27 Mar '09

There was a discussion in ticket 901 on some of the statistical tests in scipy.stats, and I thought, I post some notes that I keep to have an overview on the status of it. This doesn't cover all of stats (e.g. not descriptive statistics) Josef Inferential Statistics ====================== tests for location: ------------------- t-tests and similar ttest_1samp ttest_ind ttest_rel f_oneway (F-test) glm Notes ----- the t-tests, ttest_1samp, ttest_ind and ttest_rel, have been rewritten and are well tested glm has very incomplete description, just a t-test, needs rewrite f_oneway: verified with Nist test set for balanced anova, correct but looses numerical precision at medium to high difficulty, I have a rewrite with higher numerical precision rank based tests (some are equivalent) mannwhitneyu ranksums wilcoxon kruskal friedmanchisquare Notes ----- For 2 random variables and no ties mannwhitneyu, ranksums and kruskal are equivalent (i.e. return the same p-values but based on different statistics) kruskal has correct tie handling and works for more than two random variables friedmanchisquare has been verified, (corrected tie handling) mannwhitneyu: corrected, verified ranksums: no tie handling look at Monte Carlo p-values again, initial trying out didn't show improvement tests for scale: ---------------- ansari bartlett levene fligner mood Notes ----- I didn't verify any of them by comparing to R or matlab Brief checking with Monte Carlo shows that they work (reject wrong Null, accept correct Null) tests for distribution: ----------------------- general chisquare kstest ks_2samp anderson Notes ----- kstest, ks_2samp were rewritten and verified anderson may be fishy, but didn't look very carefully chisquare: I use a copy of it in test of discrete distributions and seems to work well for normal distribution skewtest kurtosistest normaltest shapiro Notes ----- not verified but look ok in brief Monte Carlo tests and use in examples other binom_test Notes ----- no idea Anova Ftests ------------ f_oneway (for the following: no statistics from data calculated, no p-values returned) f_value_wilks_lambda f_value f_value_multivariate Notes ----- f_oneway see above others no idea Correlation measures including pvalues -------------------------------------- pearsonr spearmanr pointbiserialr kendalltau Notes ----- pearsonr is just standard corrcoef, can be rewritten (mostly delegated to numpy.corrcoef) spearmanr needs rewriting, no tiehandling yet, can be reduced to corrcoef on rankdata pointbiserialr can be reduced to np.corrcoef, dropped? kendalltau is verified, p-value (variance) does not correct for ties extension in cython attached to ticket (but no p-values) Distributions - diagnostics and graphical analysis ================================================== box-cox transformation only checked whether they run plots look ok, converted to matplotlib pdfapprox is broken, I have enhanced rewrite, no good tests yet

1 0

Re: [SciPy-user] Monotone Interpolation
by Darcoux Christine 27 Mar '09

27 Mar '09

Any hints on how to do this with scipy ? Should I ask dev to implement it ?

2 1

"Advanced Scientific Programming in Python" Summer School in Berlin. Germany
by Tiziano Zito 27 Mar '09

27 Mar '09

Hi all, the summer school I'm co-organizing may be of interest to some of you... cheers, tiziano ------------------- "Advanced Scientific Programming in Python" a G-Node Summer School Many scientists spend much of their time writing, debugging, and maintaining software. But while techniques for doing this efficiently have been developed, only few scientists actually use them. As a result, they spend far too much time writing deficient code and reinventing the wheel instead of doing research. In this course we present a selection of advanced programming techniques with theoretical lectures and practical exercises tailored to the needs of the programming scientist. To spice up theory and foster our new skills in a real-world programming project, we will team up to develop an entertaining scientific computer game. We will use the Python programming language for the entire course. With a large collection of open-source scientific modules and all features of a full-fledged programming language, Python is rapidly gaining popularity in the neuroscience community. It enables the scientist to quickly develop powerful, efficient, and structured software and is becoming an essential tool for scientific computing. The summer school is targeted at Post-docs and PhD students from all areas of neuroscience. Substantial proficiency in Python or in another language (e.g. Java, C/C++, MATLAB, Mathematica) is absolutely required. An optional, one-day pre-course is offered to participants without Python experience to familiarize with the language. Date and Location ----------------- August 31st, 2009 -- September 4th, 2009. Berlin, Germany. Preliminary Program ------------------- Day 0 (Mon Aug 31) -- [Optional] Dive into Python Day 1 (Tue Sep 1) -- Software Carpentry - Documenting code and using version control - Test-driven development & unit testing - Debugging, profiling and benchmarking techniques - Object-oriented programming, design patterns and Extreme Programming Day 2 (Wed Sep 2) -- Scientific Tools for Python - NumPy, SciPy, Matplotlib, IPython - Neuroscience libraries - Programming project in the afternoon Day 3 (Thu Sep 3) -- Parallelization - Python multiprocessing for SMP machines - Distributed parallelization for cluster computing - Programming project in the afternoon Day 4 (Fri Sep 4) -- Practical Software Development - Software design - Efficient programming in teams - Quality Assurance - Finalizing the programming project Applications ------------ Applications should be sent before May 31st, 2009 to pythonsummerschool(a)bccnberlin.de. No fee is charged but participants should take care of travel, living, and accommodation expenses. Applications should include full contact information (name, affiliation, email & phone), a short CV and a short statement addressing the following questions (maximum 500 words): - What is your educational background? - What experience do you have in programming? - Why do you think "Advanced Scientific Programming in Python" is an appropriate course for your skill profile? Candidates will be selected based on their profile. Places are limited: early application is recommended. Faculty ------- Pietro Berkes, Volen Center for Complex Systems, Brandeis University, USA Jens Kremkow, Institut de Neurosciences Cognitives de la Méditerranée, CNRS, Marseille, France Eilif Muller, Laboratory of Computational Neuroscience, Ecole Polytechnique Fédérale de Lausanne, Switzerland Michael Schmuker, Neurobiology, Freie Universität Berlin, Germany Bartosz Telenczuk, Charité Universitätsmedizin Berlin, Germany Niko Wilbert, Institute for Theoretical Biology, Humboldt-Universität zu Berlin, Germany Tiziano Zito, Bernstein Center for Computational Neuroscience Berlin, Germany Organized by Michael Schmuker and Tiziano Zito for the German Neuroinformatics Node of the INCF. Website: http://www.g-node.org/Teaching Contact: python-summerschool(a)bccn-berlin.de

2 2

Automatic Differentiation with PYADOLC and Removing Boost::Python dependency
by Sebastian Walter 27 Mar '09

27 Mar '09

Hello, I have implemented a wrapper for the C++ Automatic Differentiation (AD) tool ADOL-C. You can use it to differentiate complex algorithms to arbitrary order. It works quite well with numpy. You can have a look at it at http://github.com/b45ch1/pyadolc . EXAMPLE USAGE: ============== compute the Jacobian J of f(x) = numpy.dot(A,x), where A is an (N,M) array --------------- get_started.py ---------------------- import numpy from adolc import * N = M = 10 A = numpy.zeros((M,N)) A[:] = [[ 1./N +(n==m) for n in range(N)] for m in range(M)] def f(x): return numpy.dot(A,x) # tape a function evaluation ax = numpy.array([adouble(0) for n in range(N)]) trace_on(1) independent(ax) ay = f(ax) dependent(ay) trace_off() x = numpy.array([n+1 for n in range(N)]) # compute jacobian of f at x J = jacobian(1,x) # compute gradient of f at x if M==1: g = gradient(1,x) --------------- end get_started.py ---------------------- PERFORMANCE: ============= It is really fast compared to existing AD tools for Python as for example Scientific.Functions.Derivatives. Benchmark available at http://github.com/b45ch1/pyadolc/blob/239a18c773c19a71bb5508dee175473a2fad7… compute hessian of: def f(x): return 0.5*dot(x,dot(A,x)) Runtime comparison: adolc: elapsed time = 0.000411 sec Scientific: elapsed time = 0.041264 sec ratio time adolc/Scientific Python: 0.009961 I.e. pyadolc is a factor 100 faster. Removing Boost::Python dependency ? =============================== I have used Boost::Python to wrap it, but I am not happy with that additional dependency! So I wondered if someone could give me advice how to avoid users having to download and install boost::python to use pyadolc. (include boost::python sources ? port to C API?) best regards, Sebastian Walter

8 17

optimize.leastsq confusing when it comes to errors
by Pim Schellart 27 Mar '09

27 Mar '09

Dear Scipy users, This is my first post to the list so please let me know if I am posting the question in the wrong place. I need to fit several functions to a dataset and would love to use scipy for this, this would allow me to use Python for all my scientific work as this is the only task for which I still use gnuplot. But I find the documentation on optimize.leastsq very confusing. I can get the fit paramaters (using the example as a guide) but I also need the errors on the resulting parameters and basically all the information given by the default gnuplot fit command output, which is the following. ################################################################### final sum of squares of residuals : 1588.68 rel. change during last iteration : -2.92059e-06 degrees of freedom (FIT_NDF) : 27 rms of residuals (FIT_STDFIT) = sqrt(WSSR/ndf) : 7.67072 variance of residuals (reduced chisquare) = WSSR/ndf : 58.84 Final set of parameters Asymptotic Standard Error ======================= ========================== a = 1.77246 +/- 1.984 (111.9%) b = 0.713972 +/- 0.06909 (9.677%) c = -0.207495 +/- 1.413 (681.2%) correlation matrix of the fit parameters: a b c a 1.000 b 0.004 1.000 c -0.103 -0.088 1.000 ################################################################### Furthermore my datapoints have associated errors and these need to be taken into account in the fit. How can I make sure this is done? Is optimize.leastsq build to do this or should I use a different function. It would be even better if the example in the tutorial is extended to include this information as I can imagine most real world applications need errors on both the input and the output. Thank you very much in advance. Kind regards, Pim Schellart P.S. I am aware of the gnuplot python module but I prefer scipy :)

3 2

Re: [SciPy-user] Loopless square 2d arrays with radially symmetric functions
by David Baddeley 26 Mar '09

26 Mar '09

There's always: X, Y = numpy.mgrid[-128:128, -128:128] R2 = X**2 + Y**2 p = numpy.exp(-0.01*R2) which uses a bit more memory, but avoids loops. David Message: 3 Date: Wed, 25 Mar 2009 11:22:46 -0400 From: josef.pktd(a)gmail.com Subject: Re: [SciPy-user] Loopless square 2d arrays with radially symmetric functions To: SciPy Users List <scipy-user(a)scipy.org> Message-ID: <1cd32cbb0903250822s78f66a08hc423b17aff761049(a)mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 On Wed, Mar 25, 2009 at 10:58 AM, David Vine <djvine(a)gmail.com> wrote: > Hello, > > I use a lot of radially symmetric arrays in my code and I would like to know > if there is a more efficient method for creating these arrays than the > nested for-loops i am currently using. > > For example, to create a Gaussian I would currently do this (Python 2.5, > Ubuntu Intrepid): > import scipy > p = scipy.zeros((256,256)) > for i in xrange(256): > ?? for j in xrange(256): > ?????? p[i,j] = exp(-0.01*( (i-128.)**2. - (j-128.)**2.)? ) > > and my question is whether there is a more efficient 'loopless' method (i.e > involving no for-loops) ? > > Thanks in advance > David > I think this does the same with broadcasting import numpy as np i = np.arange(256) p = np.exp(-0.01*( (i[:,np.newaxis]-128.)**2. - (i[np.newaxis,:]-128.)**2.)) Josef ------------------------------ Get fit, feel better! All the help you need at Yahoo!Xtra Health - http://nz.lifestyle.yahoo.com/health

3 2

Using fmin
by Moofle 25 Mar '09

25 Mar '09

Dear All, I am a beginner to python and scipy for that matter. I am trying to program GARCH(1,1) and I am attempting to use fmin or fmin_powell to optimize the equation : = SUM(from i to N) [-ln(vi)-((ui**2)/(vi))] (where i is the index) I am trying to get some hints as to how to answer this question: 1. vi is dependent on ui and the v before it (vi-1). It is defined by this equation: vi = omega + alpha*ui-1 + beta*vi-1 (where omega, alpha, beta are the parameters I need to optimize, i is the index) I am not sure how to pass alpha, omega, beta into the fmin method. To be honest, I am not even completely sure about how that method even works even though I have looked at the documentation for hours. In particular, I dont know what the args=() is used for!! I can solve this in excel using solver and AMPL, but I sure would appreciate a few hints!! Many Thanks.

3 2