[Numpy-discussion] A random.normal function with stdev as array

[Bruce Southey]

Hi,
Can you provide more details on what you are doing, especially how you
are using this?

The one item that is not directly part of Tim's list is that some
times you need to reorder your loops  (perhaps this is part of "Think
about your algorithm"?). Loop swapping is very common to improve
performance. However, it usually requires a very clear head or someone
else to do it. Also, you can might need to break loops into pieces
where you repeat the same tasks and computations over and over.

The other aspect is to do some algebra on the calculations as the
stdev is essentially a constant so depending on how you use it you can
factor it out further. Again it all depends on what you are actually
doing with these numbers.

>From a different view, you need to be very careful with your
(pseudo)random number generator with that many samples. These have a
tendency to repeat so your random number stream is no longer random.
See the Wikipedia entry:
http://en.wikipedia.org/wiki/Pseudorandom_number_generator

If I recall correctly, the Python random number generator is a
Mersenne twister but ranlib  is not and so prone to the mentioned
problems. I do not know if SciPy adds any other generators.

Finally I would also cheat by reducing the stdev values because in
many cases you will not see a real difference between a normal with
mean zero and variance 1.0 and a normal with mean zero and variance
1.1 (especially if you are doing more than comparing distributions so
there are more sources of 'error') unless you have a really large
number of samples.


Regards
Bruce

On 4/5/06, Eric Emsellem <emsellem at obs.univ-lyon1.fr> wrote:
> Hi,
>
> I am trying to optimize a code where I derive random numbers many times
> and having an array of values for the stdev parameter.
>
> I wish to have an efficient way of doing something like:
> ##################
> stdev = array([1.1,1.2,1.0,2.2])
> result = numpy.zeros(stdev.shape, Float)
> for i in range(len(stdev)) :
>    result[i] = numpy.random.normal(0, stdev[i])
> ##################
>
> In my case,  stdev can in fact be an array of a few millions floats...
> so I really need to optimize things.
>
> Any hint on how to code this efficiently ?
>
> And in general, where could I find tips for optimizing a code where I
> unfortunately have too many loops such as "for i in range(Nbody) : "
> with Nbody being > 10^6 ?
>
> thanks!
> Eric
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>