[SciPy-Dev] GSoC: Integrating library UNU.RAN in SciPy

[Ralf Gommers]

On Wed, Jun 9, 2021 at 12:16 PM Tirth Patel <tirthasheshpatel at gmail.com>
wrote:

> Hi Ralf,
>
> On Tue, Jun 8, 2021 at 11:14 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Tue, Jun 8, 2021 at 6:53 PM Tirth Patel <tirthasheshpatel at gmail.com>
>> wrote:
>>
>>>
>>> Other Notes
>>> -----------
>>>
>>> I think NumPy 1.19 introduced a stable interface for its Generator API.
>>> Before that the RandomState API was stable. But RandomState doesn't have a
>>> Cython Interface and its Python interface is very slow to be practically
>>> useful. For instance, it takes around 7ms to sample 100,000 samples from
>>> the normal distribution using the TDR method on NumPy >= 1.19 while it
>>> takes around 911ms on NumPy == 1.18 which is around 130 times slower! Is
>>> there a plan to drop 1.16 soon and can we use the unstable Generator API
>>> from 1.17 onwards or would it too unsafe? Maybe this discussion isn't
>>> suited here but I just thought to put it out as a note.
>>>
>>
>> We can drop NumPy 1.16 right now. I'm not sure if the 1.17 C API for
>> numpy.random is usable - it was either missing some features or not present
>> at all.
>>
>
> Nice to hear that we don't need to support 1.16 now! With that, I think
> there is a possibility of using the Cython API of NumPy BitGenerator to
> speed things up. I checked out a few releases on NumPy and found out
> BitGenerator was added in 1.17 with a Cython API. All we need is the
> `next_double` and `state` member of the `bitgen_t` object which are present
> from 1.17 onwards. The only difference is that 1.17 contains the `bitgen_t`
> in `numpy.random.common` module while it was shifted to `numpy.random` from
> 1.18 onwards. I don't know if there are any known bugs in 1.17 and 1.18
> before becoming stable in 1.19. If not, we might be able to use the
> unstable NumPy Generator in 1.17 and 1.18 for our purpose. What do you
> think?
>

I think this is fine - if it helps that much, let's just try it. If it
passes all tests with the latest bugfix releases of 1.17.x and 1.18.x then
it should be okay.

Cheers,
Ralf



> What the recently added biasedurn does is a conditional compile - only use
>> that C API for NumPy >= 1.19. If the performance on <1.19 isn't completely
>> unusable, that may be a good option?
>>
>
> Yes, that's what I do right now. But I am just worried if the performance
> on <1.19 is too slow to practically rely on. Anyways, thanks for looking
> into this!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scipy-dev/attachments/20210612/d735e7ee/attachment.html>