Mailman 3 December 2012 - SciPy-Dev

Bootstrap confidence limits code
by Constantine Evans Oct. 4, 2014

Oct. 4, 2014

Hello everyone, A few years ago I implemented a scikit for bootstrap confidence limits (https://github.com/cgevans/scikits-bootstrap). I didn’t think much about it after that until recently, when I realized that some people are actually using it, and that there’s apparently been some talk about implementing this functionality in either scipy.stats or statsmodels (I should thank Randal Olson for discussing this and bringing it to my attention). As such I’ve rewritten most of the code, and written up some docstrings. The current code can do confidence intervals with basic percentile interval, bias-corrected accelerated, and approximate bootstrap confidence methods, and can also provide bootstrap and jackknife indexes. Most of it is implemented from the descriptions in Efron and Tibshirani’s Introduction to the Bootstrap, but the ABC code at the moment is a port from the modified-BSD-licensed bootstrap package for R (not the boot package) as I’m not entirely confident in my understanding of the method. And so, I have a few questions for everyone: * Is there any interest in including this sort of code in either scipy.stats or statsmodels? If so, where do people think would be the better place? The code is relatively small; at the moment it is less than 200 lines, with docstrings probably making up 100 of those lines. * Also, if so, what would need to be changed, added, and improved beyond what is mentioned in the Contributing to Scipy part of the reference guide? I’m never a fan of my own code, and imagine quite a bit would need to be fixed; I know tests will need to be added too. In addition, I have a few questions about what would be better practice for the API, and I haven’t really found a guide on best practices for Scipy: * When I started writing the code, I wrote a single function ci for confidence intervals, with a method argument to choose the method. This is easy for users, especially so that they don’t have to look through documentation to realize that BCA is the most generally useful method (at least from everything I’ve read) and that there really isn’t any reason to use many of the simpler methods. However, ABC takes different paramenters, and needs a statistic function that takes weights, which makes this single-function organization trickier. At the moment, I have a separate function for ABC. Would it be better to split up all the methods to their own functions? * ABC requires a statistic function that takes weights. I’ve noticed that things like np.average takes a weights= argument. Would it be better to require input of a stat(data,weights) function, or input of a stat(data,weights=) with weights as a named argument? The latter would be nice in terms of allowing the same function to be used for all methods, but would make it impossible to use a lambda for the function. Is there some other method of doing this entirely? * Are there any missing features that anyone thinks should be added? I apologize if much of this is answered elsewhere, I just haven’t found any of it; I also apologize if this is far too long-winded and confusing! Regards, Constantine Evans

4 3

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

64-bit sparse matrix indices
by Pauli Virtanen Feb. 18, 2013

Feb. 18, 2013

Hi, I've been looking a bit at making sparse matrices work with 64-bit indices: https://github.com/pv/scipy-work/commits/ticket/1307 The motivation is that 32-bit indices on 64-bit machines don't allow representing sparse matrices with large nnz. One option A (currently there) is to allow both int32 and int64 as indices, and use the larger one only when required by nnz. The second option B would be to just use intp for everything. The problem with A is that I'm far from certain that I found all the corner cases yet, and I'm fairly certain there are some undiscovered bugs still somewhere. The test suite doesn't yet have the level of coverage on this issue I'd be comfortable with. The problem with B is that on 64-bit systems, it it increases the memory needs of sparse matrices by about 50%. However, as a solution it's more robust and elegant. Opinions on how it should work? Pauli

3 5

Scipy.org is down + question about "topical software"
by Michael Clerx Jan. 2, 2013

Jan. 2, 2013

Dear all, First of all, scipy.org appears to be down and has been for some time today. Yesterday, when it was still up, I registered an account and tried adding a link to the topical software section but got the error "too many links". Does anybody know how/if I can fix this? kind regards, Michael

7 12

Installing multiple scipy versions for development
by Alexander Löw Dec. 28, 2012

Dec. 28, 2012

Hi, I am a newbie to scipy development, thus sorry, if the following question might sound stupid for some of you: What is best practice to install multiple scipy versions on my machine for local development? I run python under Ubuntu and have general python 2.7 installed together with scipy 0.10.1. I recently checked out the recent scipy source code. Now I want to be able to switch easily between the developer version of scipy and the stable code. How can I do this? I did not find anything in the Wiki. I guess I would need to change some PATH variables during by session, but how? Thanks for your help! Alex

4 3

Would to post a question to the scipy mailing lists
by Shuaib Osman Dec. 28, 2012

Dec. 28, 2012

Hi, I basically need a way to make scipy.spatial.delauny find_simplex method warn me when it falls back on a brute force search - as it takes several seconds longer on large triangulations. I can brute force search for the points in question via cuda which is several times faster. Thanks.

1 0

Cython on build-time (except on source tarballs)
by Pauli Virtanen Dec. 26, 2012

Dec. 26, 2012

Hi, Here's a suggestion on how to get Cython as a build-time dependency, so that building from source tarballs (sdist) doesn't require it: https://github.com/pv/scipy-work/compare/cython-builddep I'm not completely sure this is the way to implement it, but I think getting rid of the committed-in C sources would be useful for the long run. Requiring people who want to compile from Git to have Cython installed is perhaps not such a big deal, since Cython is fairly easy to install everywhere. We'll still need the committed-in generated .pyx files, though... -- Pauli Virtanen

2 1

scipy test errors
by Skipper Seabold Dec. 22, 2012

Dec. 22, 2012

I am unable to login to my trac count anymore for some reason to comment on this ticket http://projects.scipy.org/scipy/ticket/1796 But I installed master numpy/scipy yesterday on a new system, and I'm also experience a large number of errors. Not all the same as listed on the ticket. FAILED (KNOWNFAIL=17, SKIP=37, errors=48) Test output attached. Skipper

4 7

default scipy fft performance
by Paul Anton Letnes Dec. 21, 2012

Dec. 21, 2012

Dear Scipy devs, 1) thanks for doing a great job in general! 2) Why is there a huge performance difference between scipy.fft and scipy.fftpack.fft? It apperars that scipy.fft == numpy.fft: In [4]: import numpy as np In [5]: from scipy import fft In [6]: from scipy import fftpack In [7]: d = np.linspace(0, 1e3, 1e7) In [8]: %timeit np.fft.fft(d) 1 loops, best of 3: 1.29 s per loop In [9]: %timeit fft(d) 1 loops, best of 3: 1.3 s per loop In [10]: %timeit fftpack.fft(d) 1 loops, best of 3: 651 ms per loop 3) On a related note - what's the best performing python fft library/wrapper out there? I take it from some google research I did that e.g. fftw cannot be used due to the GPL licence. Cheers, Paul

5 7

Fwd: [Numpy-discussion] Support for python 2.4 dropped. Should we drop 2.5 also?
by Sturla Molden Dec. 18, 2012

Dec. 18, 2012

Does this apply for SciPy as well? Sturla Sendt fra min iPad Videresendt melding: > Fra: Charles R Harris <charlesr.harris(a)gmail.com> > Dato: 16. desember 2012 18:28:34 CET > Til: numpy-discussion <numpy-discussion(a)scipy.org> > Emne: Re: [Numpy-discussion] Support for python 2.4 dropped. Should we drop 2.5 also? > Svar til: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > > > > On Thu, Dec 13, 2012 at 10:38 AM, Charles R Harris <charlesr.harris(a)gmail.com> wrote: >> The previous proposal to drop python 2.4 support garnered no opposition. How about dropping support for python 2.5 also? > > The proposal to drop support for python 2.5 and 2.4 in numpy 1.8 has carried. It is now a todo issue on github. > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion

2 1