Mailman 3 December 2010 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

Bug in KDTree
by Benjamin Root Jan. 10, 2011

Jan. 10, 2011

Hello, I came across an issue using KDTree. I searched the bug list, and it appears that there has already been a patch submitted for review. I have added comments to the bug report and I think the patch is good to go. I am not an expert in KDTree myself, but I think the logic is sound. http://projects.scipy.org/scipy/ticket/1052 If the first node of the tree is a leafnode, then query_pairs() will fail if the points are closer than the distance threshold. This is because the algorithm is assuming that the first node will always be an innernode. By rearranging the if-statements so that a check for a leafnode is done first, you protect against such a situation. At least, that is my understanding. I welcome comments on this. Thanks, Ben Root

2 5

Re: [SciPy-Dev] Ticket created for Lomb-Scargle periodogram
by Pim Schellart Dec. 24, 2010

Dec. 24, 2010

The code I submitted is for the original Lomb-Scargle periodogram. The equations themselves were only refactored to allow the periodogram to be calculated using only a single pass through the input arrays for each frequency. It might be possible to also add code for the generalized periodogram, although that is slightly more complicated because it involves several integrals, I'll have a look. Regards, Pim Schellart On Dec 24, 2010, at 7:00 PM, scipy-dev-request(a)scipy.org wrote: > Send SciPy-Dev mailing list submissions to > scipy-dev(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request(a)scipy.org > > You can reach the person managing the list at > scipy-dev-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-Dev digest..." > > > Today's Topics: > > 1. Re: Ticket created for Lomb-Scargle periodogram > (Johann Cohen-Tanugi) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 24 Dec 2010 09:59:00 +0100 > From: Johann Cohen-Tanugi <cohen(a)lpta.in2p3.fr> > Subject: Re: [SciPy-Dev] Ticket created for Lomb-Scargle periodogram > To: SciPy Developers List <scipy-dev(a)scipy.org> > Cc: Pim Schellart <P.Schellart(a)astro.ru.nl> > Message-ID: <4D1460D4.5040909(a)lpta.in2p3.fr> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > hi there, I did not look at the code, sorry, but is it the original LS > periodogram, or the generalized one? > > best, > Johann > > On 12/23/2010 05:23 PM, Pim Schellart wrote: >> Dear SciPy developers, >> >> as requested I created an enhancement ticket (http://projects.scipy.org/scipy/ticket/1352). >> I attached FORTRAN 95, FORTRAN 77 and ANSI C versions of the subroutine, so you may pick your favorite language :) >> Just let me know what the next step (if any) should be. >> Merry Christmas! >> >> Regards, >> >> Pim Schellart >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev(a)scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> > > > ------------------------------ > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev(a)scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > End of SciPy-Dev Digest, Vol 86, Issue 20 > *****************************************

1 0

Ticket created for Lomb-Scargle periodogram
by Pim Schellart Dec. 24, 2010

Dec. 24, 2010

Dear SciPy developers, as requested I created an enhancement ticket (http://projects.scipy.org/scipy/ticket/1352). I attached FORTRAN 95, FORTRAN 77 and ANSI C versions of the subroutine, so you may pick your favorite language :) Just let me know what the next step (if any) should be. Merry Christmas! Regards, Pim Schellart

2 1

ANN: scikits.learn 0.6
by Fabian Pedregosa Dec. 23, 2010

Dec. 23, 2010

I am very pleased to announce the 0.6 release of scikits.learn, a Python module for machine learning. Documentation: http://scikit-learn.sourceforge.net Git: http://github.com/scikit-learn/scikit-learn/ Mailing list: https://lists.sourceforge.net/lists/listinfo/scikit-learn-general Installation --------- Source packages and precompiled windows binaries can be downloaded from sourceforge: https://sourceforge.net/projects/scikit-learn/files/ or installed through pip: pip install -U scikits.learn Changelog ---------- A complete changelog for this release can be found here: http://scikit-learn.sourceforge.net/whats_new.html Best regards, Fabian.

1 0

Lomb-Scargle periodogram
by Pim Schellart Dec. 22, 2010

Dec. 22, 2010

Dear SciPy developers, in my research field of Astronomy we often encounter timeseries with uneven temporal sampling. Such timeseries cannot be analyzed with the standard Fast Fourier Transform. The most frequently used tool to analyze such timeseries is the Lomb-Scargle periodogram, developed by Lomb (1976) and further extended by Scargle (1982). Currently, to my knowledge, SciPy does not include a routine to calculate such periodograms. Therefore I would like to contribute the following code which efficiently calculates the periodogram. It is licensed under the terms of the BSD open source license but I would be willing to release it under any other language if this is necessary for possible inclusion with SciPy. The routine is written in Fortran 95 and wrapped with f2py, but I would be willing to rewrite it in C if required. Also included is a Python test script. Please provide me with any feedback in case you would consider including this. Kind regards, Pim Schellart

3 2

ANN: SciPy 0.9.0 beta 1
by Ralf Gommers Dec. 21, 2010

Dec. 21, 2010

Hi, I am pleased to announce the availability of the first beta of SciPy 0.9.0. This will be the first SciPy release to include support for Python 3, as well as for Python 2.7. Please try this beta and report any problems on the scipy-dev mailing list. Binaries, sources and release notes can be found athttp://sourceforge.net/projects/scipy/files/scipy/0.9.0b1/<https://sourceforge.net/projects/numpy/files/>. Note that not all binaries (win32-py27, *-macosx10.3) are uploaded yet, they will follow in the next day or two. There are still a few known issues (so no need to report these): 1. Arpack related errors on 64-bit OS X. 2. Correlate complex192 errors on Windows. 3. correlate/conjugate current behavior is deprecated and should be removed before RC1. Enjoy, Ralf

7 18

Re: [SciPy-Dev] ANN: SciPy 0.9.0 beta 1
by Ralf Gommers Dec. 20, 2010

Dec. 20, 2010

<resending, didn't get through first time> On Thu, Dec 16, 2010 at 9:14 PM, Ralf Gommers <ralf.gommers(a)googlemail.com>wrote: > > > On Wed, Dec 15, 2010 at 1:52 AM, Bruce Southey <bsouthey(a)gmail.com> wrote: > >> >> There is a print statement in the function test_cg for 'conjugate >> gradient optimization routine' that needs to be commented out: >> See line 70 of scipy-0.9.0b1/scipy/optimize/tests/test_optimize.py: >> 'print self.funccalls, self.gradcalls' >> >> There is a PendingDeprecationWarning with Python2.7 which I can add a >> ticket for if this is non-trivial. Relevant line from verbose=10: >> >> binary dilation 29 ... >> /usr/lib64/python2.7/site-packages/scipy/ndimage/morphology.py:254: >> PendingDeprecationWarning: The CObject type is marked Pending >> Deprecation in Python 2.7. Please use capsule objects instead. >> structure, mask, output, border_value, origin, invert, cit, 1) >> ok >> >> Is it possible to silence the 'ComplexWarning: Casting complex values >> to real discards the imaginary part' message and perhaps the >> 'Warning: divide by zero encountered in log' message while running the >> tests? My reasoning is that these tend to hide real problems. >> > > I plan to work on this before RC1, but I won't have time for it till at > least the second week of January. So if anyone wants to make a start, please > go for it. > > Cheers, > Ralf > >

1 0

error in scipy 9.0b1 build, not sure what test fails (maybe stupid question)
by Vincent Davis Dec. 12, 2010

Dec. 12, 2010

Ok so maybe this is a stupid question but I am not sure how to know what scipy test is failing from this error. (numpy.test('full') runs clean) >>> scipy.test(verbose=2) Running unit tests for scipy NumPy version 1.5.1 NumPy is installed in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy SciPy version 0.9.0b1 SciPy is installed in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy Python version 2.7.1 (r271:86882M, Nov 30 2010, 09:39:13) [GCC 4.0.1 (Apple Inc. build 5494)] nose version 0.10.4 test_linkage_cophenet_tdist_Z (test_hierarchy.TestCopheneticDistance) Tests cophenet(Z) on tdist data set. ... ok <snip> Need to import PIL for this test test_imresize3 (test_pilutil.TestPILUtil) ... SKIP: Skipping test: test_imresize3 Need to import PIL for this test Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/nosetester.py", line 327, in test t = NumpyTestProgram(argv=argv, exit=False, plugins=plugins) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/core.py", line 219, in __init__ argv=argv, testRunner=testRunner, testLoader=testLoader) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 95, in __init__ self.runTests() File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/noseclasses.py", line 349, in runTests self.result = self.testRunner.run(self.test) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/core.py", line 62, in run test(result) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 138, in __call__ return self.run(*arg, **kw) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 168, in run test(orig) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 138, in __call__ return self.run(*arg, **kw) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 168, in run test(orig) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 138, in __call__ return self.run(*arg, **kw) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 168, in run test(orig) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 138, in __call__ return self.run(*arg, **kw) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 168, in run test(orig) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 138, in __call__ return self.run(*arg, **kw) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 168, in run test(orig) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 138, in __call__ return self.run(*arg, **kw) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 158, in run result.addError(self, self.exc_info()) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/proxy.py", line 112, in addError self.assertMyTest(test) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/proxy.py", line 97, in assertMyTest % (self.test, id(self.test), test, id(test))) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py", line 129, in __repr__ unittest._strclass(self.__class__), AttributeError: 'module' object has no attribute '_strclass' -- Thanks Vincent Davis 720-301-3003

1 0

fwrap refactor: What about complex -> real?
by Dag Sverre Seljebotn Dec. 9, 2010

Dec. 9, 2010

When faced with Fortran functions expecting a real scalar, f2py will happily accept complex numbers and silently discard the imaginary part. Is this really the preferred behaviour? It would take quite a bit of work to have fwrap behave like that, and I've heard occasional complaints about the behaviour earlier (can't say I'm fond of it myself either). Opinions? (Note that Python implements ".real" on float objects, so the fix is in all cases trivial when this breaks things for people, one just appends ".real" when calling the function, which should also increase the readability of the code.) Dag Sverre

3 2