[Numpy-discussion] mtrand.c update 1.11 breaks my crappy code

Neal Becker ndbecker2 at gmail.com
Wed Apr 6 11:17:20 EDT 2016


Nathaniel Smith wrote:

> On Apr 6, 2016 06:31, "Robert Kern" <robert.kern at gmail.com> wrote:
>>
>> On Wed, Apr 6, 2016 at 2:18 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
>> >
>> > I have C++ code that tries to share the mtrand state.  It unfortunately
>> > depends on the layout of RandomState which used to be:
>> >
>> > struct __pyx_obj_6mtrand_RandomState {
>> >   PyObject_HEAD
>> >   rk_state *internal_state;
>> >   PyObject *lock;
>> > };
>> >
>> > But with 1.11 it's:
>> > struct __pyx_obj_6mtrand_RandomState {
>> >   PyObject_HEAD
>> >   struct __pyx_vtabstruct_6mtrand_RandomState *__pyx_vtab;
>> >   rk_state *internal_state;
>> >   PyObject *lock;
>> >   PyObject *state_address;
>> > };
>> >
>> > So
>> > 1. Why the change?
>> > 2. How can I write portable code?
>>
>> There is no C API to RandomState at this time, stable, portable or
> otherwise. It's all private implementation detail. If you would like a
> stable and portable C API for RandomState, you will need to contribute one
> using PyCapsules to expose the underlying rk_state* pointer.
>>
>> https://docs.python.org/2.7/c-api/capsule.html
> 
> I'm very wary about the idea of exposing the rk_state pointer at all. We
> could have a C API to random but my strong preference would be for
> something that only exposes opaque function calls that take a RandomState
> and return some random numbers, and getting even this right in a clean and
> maintainable way isn't trivial.
> 
> Obviously another option is to call one of the python methods to get an
> ndarray and read out its memory contents. If you can do this in a batch
> (fetching a bunch of numbers for each call) to amortize the additional
> overhead of going through python, then it might work fine. (Python
> overhead is not actually that much -- mostly just having to do a handful
> of extra allocations.)
> 
> Or, possibly the best option, one could use one of the many fine C random
> libraries inside your code, and if you need your code to be deterministic
> given a RandomState you could derive your state initialization from a
> single call to some RandomState method.
> 
> -n

I prefer to use a single instance of a RandomState so that there are 
guarantees about the independence of streams generated from python random 
functions, and from my c++ code.  True, there are simpler approaches - but 
I'm a purist.

Yes, if there were an api use mkl random functions from a RandomState object 
that would solve my problem.  Or even if there was an API to get a 
internal_state pointer from a RandomState object.




More information about the NumPy-Discussion mailing list