Mailman 3 September 2007 - SciPy-User

problem with fmin_cobyla
by Josh Gottlieb Sept. 7, 2007

Sept. 7, 2007

Hey, A bit of a newbie to this, but I have a problem which requires a dynamic set of constraints (some of which are non-linear) and I tried two versions using fmin_cobyla (both examples attached)-- one generates these constraints using lambda functions, but fmin seems to violate them (example in code). Then I tried generating them on the fly using exec on strings of functions, which observed the constraints, but failed to find the most optimal solution. (the third permutation should be higher) Can anyone help? Could not find any examples online which were more than trivial, and the docs dont seem very good. (apologies for the cryptic coding, I tried to minimize a real-world example into a shorter script) Thanx in advance, Josh ____________________________________________________________________________________ Pinpoint customers who are looking for what you sell. http://searchmarketing.yahoo.com/ import calendar import datetime import numpy from scipy.optimize import fmin_cobyla def Solve(): maxS = 17000000 dates = [datetime.date(2008,4,1),datetime.date(2008,5,1), datetime.date(2008,6,1)] vals = [.024925574678, .128886905103,.0447355248121] # this is a set of date lists, each one corresponding to one of the vals above and one of the unknown vars to solve for permutes = [[dates[1],dates[0]],[dates[2],dates[0]],[dates[2],dates[1]]] # daily limits (in the constraints, we use these at a monthly level) upLim = [114033,114033,114033] dnLim = [159646,159646,159646] allCons = genCons( dates,permutes,0,maxS,upLim,dnLim ) k = fmin_cobyla(minFunc, [5000000 for x in permutes], allCons, args=(permutes, vals),consargs=(),rhobeg=10000,rhoend=500,iprint=3,maxfun=100000) print permutes, vals # change to max and print print -1 * minFunc(k,permutes,vals) return k def minFunc( x, permutes, vals ): # we really need max, so we multiply by -1 return sum([-1*x[k]*vals[k] for k in xrange(len(permutes))]) def genCons( dates, permutes, initial, maxS, upLim, dnLim ): # generate constraints dynamically since normally we dont know how many dates there are maxSet = [] minSet = [] iSet = [] wSet = [] signSet = [] upperSet = [] lowerSet = [] fullRelList = [] fullSignList = [] for i,d in enumerate( dates ): iLimit = upLim[i] * ( calendar.mdays[d.month] + ( calendar.isleap(d.year) and d.month == 2 ) ) wLimit = dnLim[i] * ( calendar.mdays[d.month] + ( calendar.isleap(d.year) and d.month == 2 ) ) relList = [ n for n,t in enumerate( permutes ) if d in t ] signList = [ (t[0] == d and -1) or (t[1] == d and 1) for t in permutes if d in t ] #print iLimit,wLimit fullRelList.append(relList) fullSignList.append(signList) #print signList, relList iFn = lambda x: iLimit-sum([x[k]*signList[t] for t,k in enumerate(relList)]) # violated constraint!!! print iFn([6500000, 4949026, 0]) iSet.append(iFn) wFn = lambda x: sum([x[k]*signList[t] for t,k in enumerate(relList)])+wLimit wSet.append(wFn) maxFn = lambda x: maxS-initial-sum(numpy.concatenate([[x[k]*fullSignList[j][t] for t,k in enumerate(fullRelList[j])] for j in xrange(len(fullRelList))])) maxSet.append(maxFn) minFn = lambda x: sum(numpy.concatenate([[x[k]*fullSignList[j][t] for t,k in enumerate(fullRelList[j])] for j in xrange(len(fullRelList))]))-(0-initial) minSet.append(minFn) signPairs = numpy.concatenate( [ [ (x,y,relList.index(x),relList.index(y)) for y in relList if y!=x ] for x in relList ] ).tolist() for j,pair in enumerate( signPairs ): signFn = lambda x: x[pair[0]]*signList[pair[2]]*x[pair[1]]*signList[pair[3]] signSet.append(signFn) for j,item in enumerate( permutes ): iLimit = upLim[dates.index(item[0])] * (calendar.mdays[item[0].month]+(calendar.isleap(item[0].year) and item[0].month==2)) wLimit = dnLim[dates.index(item[1])] * (calendar.mdays[item[1].month]+(calendar.isleap(item[1].year) and item[1].month==2)) upperFn = lambda x: max(iLimit,wLimit) - x[j] lowerFn = lambda x: x[j] upperSet.append(upperFn) lowerSet.append(lowerFn) fullSet = iSet+wSet+maxSet+minSet+signSet+upperSet+lowerSet #print fullSet, len(fullSet) return fullSet import calendar import datetime import numpy from scipy.optimize import fmin_cobyla def Solve(): maxS = 17000000 dates = [datetime.date(2008,4,1),datetime.date(2008,5,1), datetime.date(2008,6,1)] vals = [.024925574678, .128886905103,.0447355248121] # this is a set of date lists, each one corresponding to one of the vals above and one of the unknown vars to solve for permutes = [[dates[1],dates[0]],[dates[2],dates[0]],[dates[2],dates[1]]] # daily limits (in the constraints, we use these at a monthly level) upLim = [114033,114033,114033] dnLim = [159646,159646,159646] allCons = genCons( dates,permutes,0,maxS,upLim,dnLim ) k = fmin_cobyla(minFunc, [5000000 for x in permutes], allCons, args=(permutes, vals),consargs=(),rhobeg=10000,rhoend=500,iprint=3,maxfun=100000) print permutes, vals # change to max and print print -1 * minFunc(k,permutes,vals) return k def minFunc( x, permutes, vals ): # we really need max, so we multiply by -1 return sum([-1*x[k]*vals[k] for k in xrange(len(permutes))]) def genCons( dates, permutes, initial, maxS, upLim, dnLim ): # generate constraints dynamically since normally we dont know how many dates there are maxSet = [] minSet = [] iSet = [] wSet = [] signSet = [] upperSet = [] lowerSet = [] fullRelList = [] fullSignList = [] for i,d in enumerate( dates ): iLimit = upLim[i] * ( calendar.mdays[d.month] + ( calendar.isleap(d.year) and d.month == 2 ) ) wLimit = dnLim[i] * ( calendar.mdays[d.month] + ( calendar.isleap(d.year) and d.month == 2 ) ) relList = [ n for n,t in enumerate( permutes ) if d in t ] signList = [ (t[0] == d and -1) or (t[1] == d and 1) for t in permutes if d in t ] #print iLimit,wLimit fullRelList.append(relList) fullSignList.append(signList) #print signList, relList iFn = lambda x: iLimit-sum([x[k]*signList[t] for t,k in enumerate(relList)]) # violated constraint!!! print iFn([6500000, 4949026, 0]) iSet.append(iFn) wFn = lambda x: sum([x[k]*signList[t] for t,k in enumerate(relList)])+wLimit wSet.append(wFn) maxFn = lambda x: maxS-initial-sum(numpy.concatenate([[x[k]*fullSignList[j][t] for t,k in enumerate(fullRelList[j])] for j in xrange(len(fullRelList))])) maxSet.append(maxFn) minFn = lambda x: sum(numpy.concatenate([[x[k]*fullSignList[j][t] for t,k in enumerate(fullRelList[j])] for j in xrange(len(fullRelList))]))-(0-initial) minSet.append(minFn) signPairs = numpy.concatenate( [ [ (x,y,relList.index(x),relList.index(y)) for y in relList if y!=x ] for x in relList ] ).tolist() for j,pair in enumerate( signPairs ): signFn = lambda x: x[pair[0]]*signList[pair[2]]*x[pair[1]]*signList[pair[3]] signSet.append(signFn) for j,item in enumerate( permutes ): iLimit = upLim[dates.index(item[0])] * (calendar.mdays[item[0].month]+(calendar.isleap(item[0].year) and item[0].month==2)) wLimit = dnLim[dates.index(item[1])] * (calendar.mdays[item[1].month]+(calendar.isleap(item[1].year) and item[1].month==2)) upperFn = lambda x: max(iLimit,wLimit) - x[j] lowerFn = lambda x: x[j] upperSet.append(upperFn) lowerSet.append(lowerFn) fullSet = iSet+wSet+maxSet+minSet+signSet+upperSet+lowerSet #print fullSet, len(fullSet) return fullSet def main(): return Solve()

3 2

Re: [SciPy-user] generalised eigenvalues and corresponding eigen vectors
by Perez Kawumi Sept. 7, 2007

Sept. 7, 2007

Hi, Just thought I'd try and rephrase my question. For my problem S is an 84*84 matrix and T is also an 84*84 matrix. I want to find the eigen values and vectors of this matrix. Im not very conversant with python but this should return the generalized eigen values and their corresponding eigen vectors( i think). Sorry can't post code my file is quite long Im tring to solve an equation that will give me both the eigen values and the eigen vectors. It's returning the right eigen values but not all the eigen vectors are correct. Just don't know if I'm doing this the right way. This is what im doing [eigvalues,eigvectors] = linalg.eig(S,T) Thanks Perez ----- Original Message ---- From: lorenzo bolla <lbolla(a)gmail.com> To: SciPy Users List <scipy-user(a)scipy.org> Sent: Friday, 7 September, 2007 9:03:52 AM Subject: Re: [SciPy-user] eigenvalues and eigen vectors can you post a piece of code? L. On 9/7/07, Perez Kawumi <pepe_kawumi(a)yahoo.co.uk> wrote: Hi, Im tring to solve an equation that will give me both the eigen values and the eigen vectors. It's returning the right eigen values but not all the eigen vectors are correct. Just don't know if I'm doing this the right way. This is what im doing [eigvalues,eigvectors] = linalg.eig(S,T) Thanks Perez Yahoo! Photos NEW, now offering a quality print service from just 8p a photo. _______________________________________________ SciPy-user mailing list SciPy-user(a)scipy.org http://projects.scipy.org/mailman/listinfo/scipy-user ___________________________________________________________ Yahoo! Answers - Got a question? Someone out there knows the answer. Try it now. http://uk.answers.yahoo.com/

2 1

eigenvalues and eigen vectors
by Perez Kawumi Sept. 7, 2007

Sept. 7, 2007

Hi, Im tring to solve an equation that will give me both the eigen values and the eigen vectors. It's returning the right eigen values but not all the eigen vectors are correct. Just don't know if I'm doing this the right way. This is what im doing [eigvalues,eigvectors] = linalg.eig(S,T) Thanks Perez ___________________________________________________________ Win a BlackBerry device from O2 with Yahoo!. Enter now. http://www.yahoo.co.uk/blackberry

2 1

scipy r3303: from scipy import fftpack fails
by Steve Schmerler Sept. 7, 2007

Sept. 7, 2007

Hi This is the second problem I found (with 0.7.0.dev3303): $ python -c "from scipy import fftpack" Traceback (most recent call last): File "<string>", line 1, in ? File "/usr/local/lib/python2.4/site-packages/scipy/fftpack/__init__.py", line 10, in ? from basic import * File "/usr/local/lib/python2.4/site-packages/scipy/fftpack/basic.py", line 13, in ? import _fftpack as fftpack ImportError: /usr/local/lib/python2.4/site-packages/scipy/fftpack/_fftpack.so: undefined symbol: zfftnd_fftw -- cheers, steve I love deadlines. I like the whooshing sound they make as they fly by. -- Douglas Adams

3 5

scipy r3303: test fails: AttributeError: 'test_pilutil' object has no attribute '_exc_info'
by Steve Schmerler Sept. 7, 2007

Sept. 7, 2007

Hi all I encountered 2 problems with the latest svn version (which I report in 2 posts): The first one: scipy.test() fails. Attached is the output of scipy.test(). Before installing, I removed any old build dir and install dir (site-packages/{numpy|scipy}) and everything. I can send build/install logs of numpy/scipy if needed. Versions are: In [2]: scipy.__version__ Out[2]: '0.7.0.dev3303' In [3]: scipy.__numpy_version__ Out[3]: '1.0.4.dev4026' TIA -- cheers, steve I love deadlines. I like the whooshing sound they make as they fly by. -- Douglas Adams

2 2

Solving an equation using scipy.optimize.newton
by fdu.xiaojf＠gmail.com Sept. 6, 2007

Sept. 6, 2007

Hi all, I'm trying to solve an equation f(x) = 0 with scipy.optimize.newton. However the problem isn't so simple. There are bound constraints for my equation: the equation cannot be evaluated when x is out of [Min, Max], but the root is always in the interval of [Min, Max] When newton() iterates to find a root, it sometimes try to evaluate the equation with a x out of [Min, Max], and then error occurs. How to solve this problem ? I couldn't easily find two points with different signs every time, so methods like brentq don't work here. Thanks a lot! Regards

5 6

mean of arrays...
by fred Sept. 6, 2007

Sept. 6, 2007

Hi, I want to compute the element wise mean of 2D or 3D arrays (~100). Is there a scipy function to do this ? TIA. Cheers, -- http://scipy.org/FredericPetit

5 13

ODR example code needed
by J. K. Sept. 6, 2007

Sept. 6, 2007

Hi, I am trying to use odr. I was able to get my programme to run on my data, I can see the data using pprint. Now the newbie problem: How do I use the data? I want need to calculate stuff with it, I want to save it to a file. I was trying to figure out the odr.Output, but I failed. Could someone copy and paste a working example code, so that I can work with that and the help included in the help? Thanks a lot, Jack K.

3 4

StandardError: 'NoneType' object has no attribute '_o'
by sandric ionut Sept. 6, 2007

Sept. 6, 2007

Hi: I am using Gdal 142 version on Windows XP sp2 with Numeric-24.2, SciPy 0.5.2.1, numpy 1.0.3.1. for Python 2.4. I want to use genericfunctions from scipy.ndimage. I have tried with the example provided in numarray users' manual release 1.5: >>> def fnc(iline, oline): ... oline[...] = iline[:-2] + 2 * iline[1:-1] + 3 * iline[2:] ... >>> print generic_filter1d(a, fnc, 3) [[ 3 8 14 17] [27 32 38 41] [51 56 62 65]] Everything works OK and I get the proper result, but when I try to use gdalnumeric to save the array to a image format (tif for example, or any other format supported by gdal) I get the following error: File "C:\Python24\Lib\site-packages\gdalnumeric.py", line 123, in SaveArray return driver.CreateCopy( filename, OpenArray(src_array,prototype) ) File "C:\Python24\Lib\site-packages\gdal.py", line 592, in CreateCopy target_ds = _gdal.GDALCreateCopy( self._o, filename, source_ds._o, StandardError: 'NoneType' object has no attribute '_o' How can I save an array into a image format supported by Gdal I don't know what I do wrong Thank you Ionut ____________________________________________________________________________________ Luggage? GPS? Comic books? Check out fitting gifts for grads at Yahoo! Search http://search.yahoo.com/search?fr=oni_on_mail&p=graduation+gifts&cs=bz

2 1

www.scipy.org down?
by Christian Meesters Sept. 5, 2007

Sept. 5, 2007

Hi, The scipy web page ( www.scipy.org ) seems to be down since a while. Usually this is fixed rather soon, so I'm writing just in case it got overlooked. (And I couldn't see any message here on the list related to this.) Christian

6 6