[Numpy-discussion] Quick Question about Optimization

[Robert Kern]

On Mon, May 19, 2008 at 7:30 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
> On Mon, May 19, 2008 at 5:52 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>
>> On Mon, May 19, 2008 at 6:39 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> > On Mon, May 19, 2008 at 4:36 PM, Robert Kern <robert.kern at gmail.com>
>> > wrote:
>> >>
>> >> On Mon, May 19, 2008 at 5:27 PM, Charles R Harris
>> >> <charlesr.harris at gmail.com> wrote:
>> >> > The latest versions of Matlab use the ziggurat method to generate
>> >> > random
>> >> > normals and it is faster than the method used in numpy. I have
>> >> > ziggurat
>> >> > code
>> >> > at hand, but IIRC, Robert doesn't trust the method ;)
>> >>
>> >> Well, I outlined the tests that would satisfy me, but I don't think
>> >> you ever responded.
>> >>
>> >>
>> >>  http://projects.scipy.org/pipermail/scipy-dev/2005-December/004405.html
>> >>  which references
>> >>
>> >>  http://projects.scipy.org/pipermail/scipy-dev/2005-December/004400.html
>> >
>> > It's been tested in the literature.
>>
>> And it happened to fail such tests. Hence the Doornik paper which
>> improves Marsaglia's method to pass the appropriate tests.
>
> Exactly. Doornik was more careful about using independent samples and also
> used a better random number generator (MWC8222), not exactly rocket
> science.  Believe it or not, I had read Doornik's paper before I did my
> implementation.

Good! I believe this is the first time you've mentioned it.

> I also used a better ziggurat, IMHO, than Marsaglia and
> Doornik.

Ah. HOs need testing. dieharder is probably de rigeur these days, and
it will accept input from a file.

  http://www.phy.duke.edu/~rgb/General/dieharder.php

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco