restricting optimize.leastsq to positive results
Hi all, I am using optimize.leastsq to fit my experimental data. Is there a way to restrict the results to be a positive float? e.g. my target function is a simple linear combination of three variables to fit f(x) : return a*A(x) + b*B(x) From my experiment I know that a,b,A,B can only be positive floats. How do I put this extra information in optimize.leastsq? Thanks for your help, Christoph
Hi Christoph, If you want to keep using the leastsq code then the way to do it is to penalize negative values via your residual. This needs to be done in a way that makes the algorithm think that the optimal solution is only in the positive half-space. So when you write your residual function as a python function, you have various options. A really simple one that breaks some of the assumptions of smoothness (i.e. for people who don't care to retain theoretical conditions on the convergence properties of leastsq, esp. for such a simple problem as this...) might look something like this (off the top of my head): from numpy import any from numpy.linalg import norm # a global algorithmic parameter that should be fairly large, depending on the scale of your residual function res(p) neg_penalty = 100 def res(p): # this is your original unconstrained residual based on distance of your data points from your fitted line return <your residual as a function of the parameters p> def residual(p): # Non-negativity pseudo-constraint # Assume p = array([a, b, A, B]) are floats, chosen by leastsq if any(p<0): return neg_penalty * norm(p[p<0]) * res(p) else: return res(p) I haven't tested this exact code, but I use similar constructions all the time. By factoring in the magnitude of negativity in the parameters you give the algorithm some information about a gradient so that it better understands how to find a better estimate. The assumption here is that you have the generic case with isolated solutions to your problem, otherwise leastsq might track towards back from a negative to a zero value for one or more parameters. This might not be what you want to happen, but there are fancier ways of keeping it from even *near* zero (like adding a very steep exponential-like function that increases as p values get close to zero). Put in print statements to see the norm of the values being returned in each if statement branch to diagnose whether you're using an appropriate value for neg_penalty. There are undoubtedly more elegant ways to do this with python than use a global like this, for instance you can pass additional parameters properly through the call to residual (see the doc string for leastsq). BTW your posting was unclear about whether A and B are scalar functions of x or just floats. If they are functions you'll have work just a little harder in your residual function to ensure A(x), B(x) stay positive. Let me know if this works out for you. Rob On 31/07/07, Christoph Rademacher <rademacherchristoph@googlemail.com> wrote:
Hi all,
I am using optimize.leastsq to fit my experimental data. Is there a way to restrict the results to be a positive float? e.g. my target function is a simple linear combination of three variables to fit f(x) : return a*A(x) + b*B(x)
From my experiment I know that a,b,A,B can only be positive floats. How do I put this extra information in optimize.leastsq?
Thanks for your help,
Christoph
Christoph Rademacher wrote:
Hi all,
I am using optimize.leastsq to fit my experimental data. Is there a way to restrict the results to be a positive float? e.g. my target function is a simple linear combination of three variables to fit f(x) : return a*A(x) + b*B(x)
From my experiment I know that a,b,A,B can only be positive floats.
What are those three variables mentioned? I can't understood are A(x), B(x) some funcs from x or something else? if you mean you have constraints a>0, b>0, A(x)>0, B(x)>0 so you have optimization problem with non-linear constraints. You should either use penalties, as it was mentioned in other letter, or directly provide these constraints to optimization solver, like optimize.cobyla, or cvxopt (GPL), or openopt (BSD) constrained NLP solver lincher (this one is still very primitive for now and requires cvxopt installed because of qp solver). If you will decide to use penalties, openopt ralg solver would be a good choice (no extern dependences), it is capable of handling very large penalties; also, you can use gradient/subgradient info and not only least squares but least abs values as well (ralg is capable of handling non-smooth problems). So way 1 is something like sum_j(a*A(xj) + b*B(xj) -Cj)^2-> min subjected to a>=0 A(x)>=0 b>=0 B(x)>=0 way 2 is sum_j(a*A(xj) + b*B(xj) -Cj)^2 + N1*a + N2*b+ N3*A(x)+N4*B(x) -> min Also, you can replace some or all of a, b, A(x), B(x) to either abs(var) or var^2 for example sum_j(a1^2 * A(xj) + b1^2 * B(xj) -Cj)^2-> min subj.to A(x)>=0 B(x)>=0 and then after solving a = a1^2 b = b1^2 if you will use ralg, I guess abs would yield more good result than ^2 another one approach would be a1 = exp(a), this func is also always positive, but it will not work well if your solution is close to zero.
How do I put this extra information in optimize.leastsq?
It's impossible, optimize.leastsq is for unconstrained problems only
Thanks for your help,
Christoph
HTH, D.
_______________________________________________ SciPy-user mailing list SciPy-user@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-user
participants (3)
-
Christoph Rademacher -
dmitrey -
Rob Clewley