[Numpy-discussion] NumPy C API question

Stefan Seefeld stefan at seefeld.name
Wed May 21 19:44:50 EDT 2014


Hi Nathaniel,

thanks for the prompt and thorough answer. You are entirely right, I
hadn't thought things through properly, so let me back up a bit.

I want to provide Python bindings to a C++ library I'm writing, which is
based on vector/matrix/tensor data types. In my naive view I would
expose these data types as NumPy arrays, creating PyArrayObject
instances as "wrappers", i.e. who borrow raw pointers to the storage
managed by the C++ objects. To make things slightly more interesting,
those C++ objects have their own storage management mechanism, which
allows data to migrate across different address spaces (such as from
host to GPU-device memory), and thus whether the host storage is valid
(i.e., contains up-to-date data) or not depends on where the last
operation was performed (which is controlled by an operation dispatcher
that is part of the library, too).

It seems if I let Python control the data lifetime, and borrow the data
temporarily from C++ I may be fine. However, I may want to expose
pre-existing C++ objects into Python, though, and it sounds like that
might be dangerous unless I am willing to clone the data so the Python
runtime can hold on to that even after my C++ runtime has released
theirs. But that changes the semantics, as the Python runtime no longer
sees the same data as the C++ runtime, unless I keep the two in sync
each time I cross the language boundary, which may be quite a costly
operation...

Does all that sound sensible ?

It seems I have some more design to do.

Thanks,
        Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...




More information about the NumPy-Discussion mailing list