[Numpy-discussion] mtrand.c update 1.11 breaks my crappy code

Nathaniel Smith njs at pobox.com
Wed Apr 6 10:31:32 EDT 2016


On Apr 6, 2016 06:31, "Robert Kern" <robert.kern at gmail.com> wrote:
>
> On Wed, Apr 6, 2016 at 2:18 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
> >
> > I have C++ code that tries to share the mtrand state.  It unfortunately
> > depends on the layout of RandomState which used to be:
> >
> > struct __pyx_obj_6mtrand_RandomState {
> >   PyObject_HEAD
> >   rk_state *internal_state;
> >   PyObject *lock;
> > };
> >
> > But with 1.11 it's:
> > struct __pyx_obj_6mtrand_RandomState {
> >   PyObject_HEAD
> >   struct __pyx_vtabstruct_6mtrand_RandomState *__pyx_vtab;
> >   rk_state *internal_state;
> >   PyObject *lock;
> >   PyObject *state_address;
> > };
> >
> > So
> > 1. Why the change?
> > 2. How can I write portable code?
>
> There is no C API to RandomState at this time, stable, portable or
otherwise. It's all private implementation detail. If you would like a
stable and portable C API for RandomState, you will need to contribute one
using PyCapsules to expose the underlying rk_state* pointer.
>
> https://docs.python.org/2.7/c-api/capsule.html

I'm very wary about the idea of exposing the rk_state pointer at all. We
could have a C API to random but my strong preference would be for
something that only exposes opaque function calls that take a RandomState
and return some random numbers, and getting even this right in a clean and
maintainable way isn't trivial.

Obviously another option is to call one of the python methods to get an
ndarray and read out its memory contents. If you can do this in a batch
(fetching a bunch of numbers for each call) to amortize the additional
overhead of going through python, then it might work fine. (Python overhead
is not actually that much -- mostly just having to do a handful of extra
allocations.)

Or, possibly the best option, one could use one of the many fine C random
libraries inside your code, and if you need your code to be deterministic
given a RandomState you could derive your state initialization from a
single call to some RandomState method.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20160406/b9233f8c/attachment.html>


More information about the NumPy-Discussion mailing list