I just wanted to follow up on the C++ side of OP email - Cython has quite a few difficulties working with C++ code at the moment. It's really more of a C solution most of the time and you must split things up into a mostly C call interface (that is the C code Cython can call) and limit exposure/complications with templates and complex C++11+ constructs. This may change in the longer term but in the near, that is the state. I used to use Boost.Python but I'm getting my feet wet with Pybind (which is basically the same api but works more as you expect it to with it's signature/type plumbing (including std::shared_ptr islanding), with some other C++11 based improvements, and is header only + submodule friendly!). I also remembered ndarray thanks to Neal's post but I haven't figured out how to leverage it better than pybind, at the moment. I'd be interested to see ndarray gain support for pybind interoperability... -Jason On Wed, Aug 31, 2016 at 1:08 PM, David Morris wrote:

On Wed, Aug 31, 2016 at 2:28 PM, Michael Bieri wrote:

...

Hi all

There are several ways on how to use C/C++ code from Python with NumPy, as given in http://docs.scipy.org/doc/numpy/user/c-info.html . Furthermore, there's at least pybind11.

I'm not quite sure which approach is state-of-the-art as of 2016. How would you do it if you had to make a C/C++ library available in Python right now?

In my case, I have a C library with some scientific functions on matrices and vectors. You will typically call a few functions to configure the computation, then hand over some pointers to existing buffers containing vector data, then start the computation, and finally read back the data. The library also can use MPI to parallelize.

I have been delighted with Cython for this purpose. Great integration with NumPy (you can access numpy arrays directly as C arrays), very python like syntax and amazing performance.

Good luck,

David

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion

Hey Ian - I hope I gave Cython a fair comment, but I have to add the disclaimer that your capability to understand/implement those solutions/workarounds in that project is greatly enhanced from your knowing the innards of Cython from being core developer on the Cython project. This doesn't detract from DyDN's accomplishments (if nothing it means Cython users should look there for how to use C++ with Cython and the workarounds used + shortcomings) but I would not expect not everyone would want to jump through those hoops to get things working without a firm understanding of Cython's edges, and all this potential for special/hack adaption code is still something to keep in mind when comparing to something more straight forward and easier to understand coming from a more pure C/C++ side, where things are a bit more dangerous and fairly more verbose but make play with the language and environment first-class (like Boost.Python/pybind). Since this thread is a survey over state and options it's my intent just to make sure readers have something bare in mind for current pros/cons of the approaches. -Jason On Wed, Aug 31, 2016 at 2:17 PM, Ian Henriksen < insertinterestingnamehere@gmail.com> wrote:

We use Cython very heavily in DyND's Python bindings. It has worked well for us even when working with some very modern C++. That said, a lot depends on exactly which C++ features you want to expose as a part of the interface. Interfaces that require things like non-type template parameters or variadic templates will often require a some extra C++ code to work them in to something that Cython can understand. In my experience, those particular limitations haven't been that hard to work with. Best, Ian Henriksen

On Wed, Aug 31, 2016 at 12:20 PM Jason Newton wrote:

...

I just wanted to follow up on the C++ side of OP email - Cython has quite a few difficulties working with C++ code at the moment. It's really more of a C solution most of the time and you must split things up into a mostly C call interface (that is the C code Cython can call) and limit exposure/complications with templates and complex C++11+ constructs. This may change in the longer term but in the near, that is the state.

I used to use Boost.Python but I'm getting my feet wet with Pybind (which is basically the same api but works more as you expect it to with it's signature/type plumbing (including std::shared_ptr islanding), with some other C++11 based improvements, and is header only + submodule friendly!). I also remembered ndarray thanks to Neal's post but I haven't figured out how to leverage it better than pybind, at the moment. I'd be interested to see ndarray gain support for pybind interoperability...

-Jason

On Wed, Aug 31, 2016 at 1:08 PM, David Morris wrote:

...

On Wed, Aug 31, 2016 at 2:28 PM, Michael Bieri wrote:

...

Hi all

There are several ways on how to use C/C++ code from Python with NumPy, as given in http://docs.scipy.org/doc/numpy/user/c-info.html . Furthermore, there's at least pybind11.

I'm not quite sure which approach is state-of-the-art as of 2016. How would you do it if you had to make a C/C++ library available in Python right now?

In my case, I have a C library with some scientific functions on matrices and vectors. You will typically call a few functions to configure the computation, then hand over some pointers to existing buffers containing vector data, then start the computation, and finally read back the data. The library also can use MPI to parallelize.

I have been delighted with Cython for this purpose. Great integration with NumPy (you can access numpy arrays directly as C arrays), very python like syntax and amazing performance.

Good luck,

David

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion

On Wed, Aug 31, 2016 at 3:57 PM Jason Newton wrote:

Hey Ian - I hope I gave Cython a fair comment, but I have to add the disclaimer that your capability to understand/implement those solutions/workarounds in that project is greatly enhanced from your knowing the innards of Cython from being core developer on the Cython project. This doesn't detract from DyDN's accomplishments (if nothing it means Cython users should look there for how to use C++ with Cython and the workarounds used + shortcomings) but I would not expect not everyone would want to jump through those hoops to get things working without a firm understanding of Cython's edges, and all this potential for special/hack adaption code is still something to keep in mind when comparing to something more straight forward and easier to understand coming from a more pure C/C++ side, where things are a bit more dangerous and fairly more verbose but make play with the language and environment first-class (like Boost.Python/pybind). Since this thread is a survey over state and options it's my intent just to make sure readers have something bare in mind for current pros/cons of the approaches.

-Jason

No offense taken at all. I'm actually not a Cython developer, just a frequent contributor. That said, knowing the compiler internals certainly helps when finding workarounds and building intermediate interfaces. My main point was just that, in my experience, Cython has worked well for many things beyond plain C interfaces and that workarounds (hackery entirely aside) for any missing features are usually manageable. Given that my perspective is a bit different in that regard, it seemed worth chiming in on the discussion. I suppose the moral of the story is that there's still not a clear cut "best" way of building wrappers and that your mileage may vary depending on what features you need. Thanks, Ian Henriksen