[Numpy-discussion] draft NEP for breaking ufunc ABI in a controlled way

[Charles R Harris]

On Mon, Sep 21, 2015 at 10:23 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Mon, Sep 21, 2015 at 7:29 AM, Jaime Fernández del Río
> <jaime.frio at gmail.com> wrote:
> > We have the PyArrayObject vs PyArrayObject_fields definition in
> > ndarraytypes.h that is used to enforce access to the members through
> inline
> > functions rather than directly,  which seems to me like the right way to
> go:
> > don't leave stones unturned, hide everything and provide PyUFunc_NIN,
> > PyUFunc_NOUT and friends to handle those too.
>
> The PyArrayObject vs PyArrayObject_fields distinction is only enabled
> if a downstream library explicitly requests it with #define
> NPY_NO_DEPRECATED_API, though -- the idea is that the changes in this
> NEP would be enabled unconditionally, even for old code. So the reason
> nin/nout/nargs are left exposed in this proposal is that there's some
> existing code out there that would break (until updated) if we hid
> them, and not much benefit to breaking it.
>
> If we're fine with breaking that code then we could just hide them
> unconditionally too. The only code I found in the wild that would be
> affected is the "rational" user-defined dtype, which would be
> trivially fixable since the only thing it does with ufunc->nargs is a
> quick consistency check:
>
>
> https://github.com/numpy/numpy-dtypes/blob/c0175a6b1c5aa89b4520b29487f06d0e200e2a03/npytypes/rational/rational.c#L1140-L1151
>
> Also it's not 100% clear right now whether we even want to keep
> supporting the old user-defined dtype API that this particular code is
> based around. But if this code uses ufunc->nargs then perhaps other
> code does too? I'm open to opinions -- I doubt it matters that much
> either way. I just want to make sure that we can hide the other stuff
> :-).
>
> When it comes to evolving these APIs in general: one unfortunate thing
> about the PyArrayObject changes in 1.7 is that because they were
> implemented using *inline* functions (/macros) they haven't affected
>

One thing we might consider along the way is separating numpy.multiarray
and friends into an actual library plus a module. That way the new numpy
api would be exposed in the library rather than by importing an array of
pointers from the module.

the a*B*i exposure at all, even in code that has upgraded to the new
> calling conventions. While user code no longer *names* the internal
> fields directly, we still have to implement exactly the same fields
> and put them in exactly the same place in memory or else break ABI.
> And the other unfortunate thing is that we don't really have a
> mechanism for saying "okay, we're dropping support for the old way of
> doing things in 1.xx" -- in particular the current
> NPY_NO_DEPRECATED_API mechanism doesn't give us any way to detect and
> error out if someone tries to use an old version of the APIs, so ABI
> breaks still mean segfaults. I'm thinking that if/when we figure out
> how to implement the "sliding window" API/ABI idea that we talked
> about at SciPy, then that will give us a strategy for cleanly
> transitioning to a world with a maintainable API+ABI and it becomes
> worth sitting down and making up a set of setters/getters for the
> attributes that we want to make public in a maintainable way. But
> until then our only real options are either hard breaks or nothing, so
> unless we want to do a hard break there's not much point talking about
> it.
>
> -n
>

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20150922/136a353a/attachment.html>