[SciPy-User] Avoiding inner for loops??

[eat]

Hi,

On Mon, Aug 20, 2012 at 2:47 AM, Martin De Kauwe <mdekauwe at gmail.com> wrote:

> Perhaps simplifying, in 2D this is what I want if using loops
>
> def fake_model(data1, data2, p1, p2):
>     """ complete nonsense """
>     return data1 + data2 * p1 * p2
>
> grid_size = 5
> nobs = 5
> obs = np.zeros(nobs)
> data1 = np.arange(nobs)
> data2 = np.arange(nobs)
> a = np.arange(grid_size)
> b = np.arange(grid_size)
> c = np.arange(grid_size)
> ss = np.zeros(0)
> for p1 in a:
>     for p2 in b:
>         ans = fake_model(data1, data2, p1, p2)
>         #ss = np.append(ss, np.sum(obs - ans)**2)
>         print ans
>
> Your script with a slightly modified data and parameters, will produce
[10 13 16 19 22]
[15 19 23 27 31]
[20 25 30 35 40]
snip
[100 121 142 163 184]
[125 151 177 203 229]
[150 181 212 243 274]

which is equivalent to:
In []: p1, p2= ix_(a, b)
In []: n_= newaxis
In []: ans= fake_model(data1[:, n_, n_], data2[:, n_, n_], p1, p2)
In []: ans.reshape(-1, grid_size** 2).T
Out[]:
array([[ 10,  13,  16,  19,  22],
       [ 15,  19,  23,  27,  31],
       [ 20,  25,  30,  35,  40],
snip
       [100, 121, 142, 163, 184],
       [125, 151, 177, 203, 229],
       [150, 181, 212, 243, 274]])

Regards,
-eat

>
> which would produce
>
> [0 1 2 3 4]
> [0 1 2 3 4]
> [0 1 2 3 4]
> [0 1 2 3 4]
> [0 1 2 3 4]
> [0 1 2 3 4]
> [0 2 4 6 8]
> .
> snip
> .
> [0 1 2 3 4]
> [ 0  5 10 15 20]
> [ 0  9 18 27 36]
> [ 0 13 26 39 52]
> [ 0 17 34 51 68]
>
> And so I figured something like...
>
> a = np.ones((grid_size,grid_size)) * np.arange(grid_size)[None,:]
> b = np.ones((grid_size,grid_size)) * np.arange(grid_size)[:,None]
> ans = fake_model(data1, data2, a, b)
>
> Although this doesn't seem to work, but I think this might be along the
> right lines? This produces
>
> [[  0.   1.   2.   3.   4.]
>  [  0.   2.   6.  12.  20.]
>  [  0.   3.  10.  21.  36.]
>  [  0.   4.  14.  30.  52.]
>  [  0.   5.  18.  39.  68.]]
>
>
>
>
>
>
>
>
>
> On Sunday, August 19, 2012 7:07:59 PM UTC+10, Martin De Kauwe wrote:
>>
>> Hi,
>>
>> I need to avoid (at least) two inner for loops in what I am trying to do
>> otherwise my processing takes forever. What is the best way to transfer
>> what I am doing into a more "numpy way"? Essentially I am trying to call a
>> model again for various different parameter combinations. The example is
>> fictional, by the grid_size would ideally grow > 500 and by doing so the
>> processing speed becomes very slow the way I have set things up..
>>
>> thanks.
>>
>> example.
>>
>>
>> import numpy as np
>>
>> def fake_model(data1, data2, p1, p2, p3):
>>     """ complete nonsense """
>>     return data1 + data2 * p1 * p2 * p3
>>
>> data1 = np.random.rand(10) # the size of this arrays varies might be 10
>> might be 15 etc
>> data2 = np.random.rand(10) # the size of this arrays varies might be 10
>> might be 15 etc
>> obs = np.random.rand(10) # the size of this arrays varies might be 10
>> might be 15 etc
>>
>> grid_size = 10 # Ideally this would be a large number
>> param1 = np.linspace(5.0, 350, grid_size)
>> param2 = np.linspace(5.0, 550, grid_size)
>> param3 = np.linspace(1E-8, 10.5, grid_size)
>> ss = np.zeros(0)
>>
>> for p1 in param1:
>>     for p2 in param2:
>>         for p3 in param3:
>>             ans = fake_model(data1, data2, p1, p2, p3)
>>
>>             ss = np.append(ss, np.sum(obs - ans)**2)
>>             print np.sum(obs - ans)**2
>>
>>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20120820/d335bdb2/attachment.html>