Mailman 3 random permutation - NumPy-Discussion

This has been fixed in more recent versions. http://projects.scipy.org/scipy/numpy/ticket/374 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

Keith Goodman

13 Jan 13 Jan

10:01 a.m.

On 1/11/07, Robert Kern wrote:

...

Keith Goodman wrote:

...
Why is the first element of the permutation always the same? Am I using random.permutation in the right way?

...
...
...
M.__version__ '1.0rc1'

This has been fixed in more recent versions.

http://projects.scipy.org/scipy/numpy/ticket/374

I don't see any unit tests for numpy.random. I guess randomness is hard to test. Would it help to seed the random number generator and at least check that you get the same result you got before? No existing problems would be found. But new ones might be caught. I guess the alternative is a long running, occasionally failing, statistical tests.

Sebastian Haase

10:16 a.m.

On 1/13/07, Keith Goodman wrote:

...

On 1/11/07, Robert Kern wrote:

...
Keith Goodman wrote:

...
Why is the first element of the permutation always the same? Am I using random.permutation in the right way?

...
...
...
M.__version__ '1.0rc1'

This has been fixed in more recent versions.

http://projects.scipy.org/scipy/numpy/ticket/374

I don't see any unit tests for numpy.random. I guess randomness is hard to test.

Would it help to seed the random number generator and at least check that you get the same result you got before? No existing problems would be found. But new ones might be caught.

Hi, Is it guaranteed that a given seed produces the same sequence of rand-number between different platforms ? I thought this might only be guaranteed for "any given computer" to reproduce the same numbers. -Sebastian Haase

Keith Goodman

10:26 a.m.

On 1/13/07, Sebastian Haase wrote:

...

On 1/13/07, Keith Goodman wrote:

...
On 1/11/07, Robert Kern wrote:

...
Keith Goodman wrote:

...
Why is the first element of the permutation always the same? Am I using random.permutation in the right way?

...
...
...
M.__version__ '1.0rc1'

This has been fixed in more recent versions.

http://projects.scipy.org/scipy/numpy/ticket/374

I don't see any unit tests for numpy.random. I guess randomness is hard to test.

Would it help to seed the random number generator and at least check that you get the same result you got before? No existing problems would be found. But new ones might be caught.

Hi, Is it guaranteed that a given seed produces the same sequence of rand-number between different platforms ? I thought this might only be guaranteed for "any given computer" to reproduce the same numbers.

I hope, and expect, that it is system independent. Here's what I get:

...

...
rs = numpy.random.RandomState([123, 901, 789]) rs.randn(4,1)

array([[ 0.76072026], [ 1.27712191], [ 0.03497453], [ 0.09056668]])

...

...
rs.rand(4,1)

array([[ 0.184306 ], [ 0.58967936], [ 0.52425903], [ 0.33389408]])

...

...
numpy.__version__ '1.0.1'

Linux kel 2.6.18-3-686 #1 SMP Mon Dec 4 16:41:14 UTC 2006 i686 GNU/Linux

Sebastian Haase

11:58 a.m.

On 1/13/07, Keith Goodman wrote:

...

On 1/13/07, Sebastian Haase wrote:

...
On 1/13/07, Keith Goodman wrote:

...
On 1/11/07, Robert Kern wrote:

...
Keith Goodman wrote:

...
Why is the first element of the permutation always the same? Am I using random.permutation in the right way?

...
...
> M.__version__ '1.0rc1'

This has been fixed in more recent versions.

http://projects.scipy.org/scipy/numpy/ticket/374

I don't see any unit tests for numpy.random. I guess randomness is hard to test.

Would it help to seed the random number generator and at least check that you get the same result you got before? No existing problems would be found. But new ones might be caught.

Hi, Is it guaranteed that a given seed produces the same sequence of rand-number between different platforms ? I thought this might only be guaranteed for "any given computer" to reproduce the same numbers.

I hope, and expect, that it is system independent.

Here's what I get:

...
...
rs = numpy.random.RandomState([123, 901, 789]) rs.randn(4,1)

array([[ 0.76072026], [ 1.27712191], [ 0.03497453], [ 0.09056668]])

...
...
rs.rand(4,1)

array([[ 0.184306 ], [ 0.58967936], [ 0.52425903], [ 0.33389408]])

...
...
numpy.__version__ '1.0.1'

Linux kel 2.6.18-3-686 #1 SMP Mon Dec 4 16:41:14 UTC 2006 i686 GNU/Linux import numpy rs = numpy.random.RandomState([123, 901, 789]) rs.randn(4,1) [[ 0.76072026] [ 1.27712191] [ 0.03497453] [ 0.09056668]] rs.rand(4,1) [[ 0.184306 ] [ 0.58967936] [ 0.52425903] [ 0.33389408]] numpy.__version__ '1.0rc1'

Windows XP- pentium4 - (non current numpy) - Looks promising - but how about PowerPC macs ... -Sebastian

Keith Goodman

12:10 p.m.

On 1/13/07, Sebastian Haase wrote:

...

On 1/13/07, Keith Goodman wrote:

...
On 1/13/07, Sebastian Haase wrote:

...
On 1/13/07, Keith Goodman wrote:

...
On 1/11/07, Robert Kern wrote:

...
Keith Goodman wrote:

...
Why is the first element of the permutation always the same? Am I using random.permutation in the right way?

...
>> M.__version__ '1.0rc1'

This has been fixed in more recent versions.

http://projects.scipy.org/scipy/numpy/ticket/374

I don't see any unit tests for numpy.random. I guess randomness is hard to test.

Would it help to seed the random number generator and at least check that you get the same result you got before? No existing problems would be found. But new ones might be caught.

Hi, Is it guaranteed that a given seed produces the same sequence of rand-number between different platforms ? I thought this might only be guaranteed for "any given computer" to reproduce the same numbers.

I hope, and expect, that it is system independent.

Here's what I get:

...
...
rs = numpy.random.RandomState([123, 901, 789]) rs.randn(4,1)

array([[ 0.76072026], [ 1.27712191], [ 0.03497453], [ 0.09056668]])

...
...
rs.rand(4,1)

array([[ 0.184306 ], [ 0.58967936], [ 0.52425903], [ 0.33389408]])

...
...
numpy.__version__ '1.0.1'

Linux kel 2.6.18-3-686 #1 SMP Mon Dec 4 16:41:14 UTC 2006 i686 GNU/Linux import numpy rs = numpy.random.RandomState([123, 901, 789]) rs.randn(4,1) [[ 0.76072026] [ 1.27712191] [ 0.03497453] [ 0.09056668]] rs.rand(4,1) [[ 0.184306 ] [ 0.58967936] [ 0.52425903] [ 0.33389408]] numpy.__version__ '1.0rc1'

Windows XP- pentium4 - (non current numpy) -

Looks promising - but how about PowerPC macs ...

The random numbers are generated by a fixed algorithm. So they are not random at all. As long as everyone is using float64 I would think we'd all get the same (pseudo) random numbers.

Sebastian Haase

12:28 p.m.

On 1/13/07, Keith Goodman wrote:

...

On 1/13/07, Sebastian Haase wrote:

...
On 1/13/07, Keith Goodman wrote:

...
On 1/13/07, Sebastian Haase wrote:

...
On 1/13/07, Keith Goodman wrote:

...
On 1/11/07, Robert Kern wrote:

...
Keith Goodman wrote: > Why is the first element of the permutation always the same? Am I > using random.permutation in the right way?

>>> M.__version__ > '1.0rc1'

This has been fixed in more recent versions.

http://projects.scipy.org/scipy/numpy/ticket/374

I don't see any unit tests for numpy.random. I guess randomness is hard to test.

Would it help to seed the random number generator and at least check that you get the same result you got before? No existing problems would be found. But new ones might be caught.

Hi, Is it guaranteed that a given seed produces the same sequence of rand-number between different platforms ? I thought this might only be guaranteed for "any given computer" to reproduce the same numbers.

I hope, and expect, that it is system independent.

Here's what I get:

...
...
rs = numpy.random.RandomState([123, 901, 789]) rs.randn(4,1)

array([[ 0.76072026], [ 1.27712191], [ 0.03497453], [ 0.09056668]])

...
...
rs.rand(4,1)

array([[ 0.184306 ], [ 0.58967936], [ 0.52425903], [ 0.33389408]])

...
...
numpy.__version__ '1.0.1'

Linux kel 2.6.18-3-686 #1 SMP Mon Dec 4 16:41:14 UTC 2006 i686 GNU/Linux import numpy rs = numpy.random.RandomState([123, 901, 789]) rs.randn(4,1) [[ 0.76072026] [ 1.27712191] [ 0.03497453] [ 0.09056668]] rs.rand(4,1) [[ 0.184306 ] [ 0.58967936] [ 0.52425903] [ 0.33389408]] numpy.__version__ '1.0rc1'

Windows XP- pentium4 - (non current numpy) -

Looks promising - but how about PowerPC macs ...

The random numbers are generated by a fixed algorithm. So they are not random at all. As long as everyone is using float64 I would think we'd all get the same (pseudo) random numbers.

I understand this - but I thought the algorithm might involve some rounding(-error) cases that would produce different results especially when changing the CPU (intel vs. PowerPC). I found this to be true even for non pseudo-random code that between CPU-types results were different within a given epsilon. -Sebastian

Charles R Harris

8:54 p.m.

On 1/13/07, Sebastian Haase wrote: <snip> I understand this - but I thought the algorithm might involve some

...

rounding(-error) cases that would produce different results especially when changing the CPU (intel vs. PowerPC). I found this to be true even for non pseudo-random code that between CPU-types results were different within a given epsilon.

Its all 32 bit words, or rather bits. IIRC, it is based on an irreducible polynomial over Z_2, so all the coefficients are 0 or 1. The original paper is worth a read if you like that sort of stuff. Chuck

Robert Kern

9:22 p.m.

Charles R Harris wrote:

...

On 1/13/07, *Sebastian Haase* mailto:seb.haase@gmx.net> wrote:

<snip>

I understand this - but I thought the algorithm might involve some rounding(-error) cases that would produce different results especially when changing the CPU (intel vs. PowerPC). I found this to be true even for non pseudo-random code that between CPU-types results were different within a given epsilon.

Its all 32 bit words, or rather bits. IIRC, it is based on an irreducible polynomial over Z_2, so all the coefficients are 0 or 1. The original paper is worth a read if you like that sort of stuff.

Well, there are a couple of issues, here. One is the core Mersenne Twister PRNG. You are right that it should reliably produce the same bitstream on all supported platforms. In fact, for the same state vector (*not* the seed), it should produce the same results as the standard library's random.Random() implementation. Now, the various distributions are another matter. They do rely on floating point calculations, and I doubt that we can rely on them producing the exact same streams of output on different processors, particularly those that use extended-precision intermediates. Unfortunately, running distribution tests on all of the non-uniform distributions requires implementations of lots of special functions. I did run such tests using scipy.stats when I implemented the distributions, but I can't really turn those into unit tests for numpy. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

Stefan van der Walt

1:15 p.m.

On Sat, Jan 13, 2007 at 10:01:59AM -0800, Keith Goodman wrote:

...

On 1/11/07, Robert Kern wrote:

...
Keith Goodman wrote:

...
Why is the first element of the permutation always the same? Am I using random.permutation in the right way?

...
...
...
M.__version__ '1.0rc1'

This has been fixed in more recent versions.

http://projects.scipy.org/scipy/numpy/ticket/374

I don't see any unit tests for numpy.random. I guess randomness is hard to test.

Every time we fix a bug, we add a corresponding test to make sure that it doesn't pop up again. In this case, take a look in numpy/core/tests/test_regression.py: def check_random_shuffle(self, level=rlevel): """Ticket #374""" a = N.arange(5).reshape((5,1)) b = a.copy() N.random.shuffle(b) assert_equal(sorted(b),a) Cheers Stéfan

Ted Horst

25 Jan 25 Jan

7:30 p.m.

BTW, This test doesn't work on python 2.3 because sorted does not exist there. Ted On Jan 13, 2007, at 15:15, Stefan van der Walt wrote:

...

On Sat, Jan 13, 2007 at 10:01:59AM -0800, Keith Goodman wrote:

...
On 1/11/07, Robert Kern wrote:

...
Keith Goodman wrote:

...
Why is the first element of the permutation always the same? Am I using random.permutation in the right way?

...
...
...
M.__version__ '1.0rc1'

This has been fixed in more recent versions.

http://projects.scipy.org/scipy/numpy/ticket/374

I don't see any unit tests for numpy.random. I guess randomness is hard to test.

Every time we fix a bug, we add a corresponding test to make sure that it doesn't pop up again. In this case, take a look in numpy/core/tests/test_regression.py:

def check_random_shuffle(self, level=rlevel): """Ticket #374""" a = N.arange(5).reshape((5,1)) b = a.copy() N.random.shuffle(b) assert_equal(sorted(b),a)

Cheers Stéfan _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

Robert Kern

7:47 p.m.

Ted Horst wrote:

...

BTW, This test doesn't work on python 2.3 because sorted does not exist there.

Fixed, thank you. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

6297

Age (days ago)

6312

Last active (days ago)

List overview

Download

12 comments

6 participants

participants (6)

Charles R Harris
Keith Goodman
Robert Kern
Sebastian Haase
Stefan van der Walt
Ted Horst

random permutation

Keith Goodman

Keith Goodman

Keith Goodman

Keith Goodman

Ted Horst

tags

participants (6)