[Numpy-discussion] reorganizing numpy internal extensions (was: Re: Should we drop support for "one file" compilation mode?)

[David Cournapeau]

On Thu, Oct 8, 2015 at 8:47 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Oct 8, 2015 06:30, "David Cournapeau" <cournape at gmail.com> wrote:
> >
> [...]
> >
> > Separating the pure C code into static lib is the simple way of
> achieving the same goal. Essentially, you write:
> >
> > # implemented in npyinternal.a
> > _npy_internal_foo(....)
> >
> > # implemented in merged_multiarray_umath.pyx
> > cdef PyArray_Foo(...):
> >     # use _npy_internal_foo()
> >
> > then our merged_multiarray_umath.so is built by linking the .pyx and the
> npyinternal.a together. IOW, the static link is internal.
> >
> > Going through npyinternal.a instead of just linking .o from pure C and
> Cython together gives us the following:
> >
> >  1. the .a can just use normal linking strategies instead of the awkward
> capsule thing. Those are easy to get wrong when using cython as you may end
> up with multiple internal copies of the wrapped object inside capsule,
> causing hard to track bugs (this is what we wasted most of the time on w/
> Stefan and Kurt during ds4ds)
>
> Check out Stéfan's branch -- it just uses regular linking to mix cython
> and C.
>
I know, we worked on this together after all ;)

My suggested organisation is certainly not mandatory, I was not trying to
claim otherwise, sorry if that was unclear.

At that point, I guess the consensus is that I have to prove my suggestion
is useful. I will take a few more hours to submit a PR with the umath
conversion (maybe merging w/ the work from Stéfan). I discovered on my
flight back that you can call PyModule_Init multiple times for a given
module, which is useful while we do the transition C->Cython for the module
initialization (it is not documented as possible, so I would not to rely on
it for long either).

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20151008/3eccadce/attachment.html>