[Numpy-discussion] numpy 2.0, what else to do?

[David Cournapeau]

Charles R Harris wrote:
> 
> 
> On Mon, Feb 15, 2010 at 8:13 PM, David Cournapeau <david at silveregg.co.jp 
> <mailto:david at silveregg.co.jp>> wrote:
> 
>     Charles R Harris wrote:
>      >
>      >
>      > On Mon, Feb 15, 2010 at 3:58 PM, Charles R Harris
>      > <charlesr.harris at gmail.com <mailto:charlesr.harris at gmail.com>
>     <mailto:charlesr.harris at gmail.com
>     <mailto:charlesr.harris at gmail.com>>> wrote:
>      >
>      >
>      >
>      >     On Mon, Feb 15, 2010 at 3:34 PM, David Cournapeau
>      >     <cournape at gmail.com <mailto:cournape at gmail.com>
>     <mailto:cournape at gmail.com <mailto:cournape at gmail.com>>> wrote:
>      >
>      >         On Tue, Feb 16, 2010 at 7:04 AM, Charles R Harris
>      >         <charlesr.harris at gmail.com
>     <mailto:charlesr.harris at gmail.com> <mailto:charlesr.harris at gmail.com
>     <mailto:charlesr.harris at gmail.com>>>
>      >         wrote:
>      >          >
>      >          >
>      >          > On Mon, Feb 15, 2010 at 2:46 PM, David Cournapeau
>      >         <cournape at gmail.com <mailto:cournape at gmail.com>
>     <mailto:cournape at gmail.com <mailto:cournape at gmail.com>>>
>      >          > wrote:
>      >          >>
>      >          >> On Tue, Feb 16, 2010 at 4:08 AM, Charles R Harris
>      >          >> <charlesr.harris at gmail.com
>     <mailto:charlesr.harris at gmail.com>
>      >         <mailto:charlesr.harris at gmail.com
>     <mailto:charlesr.harris at gmail.com>>> wrote:
>      >          >>
>      >          >> >
>      >          >> > I was wondering about that. Why do we have a private
>      >         include directory?
>      >          >> > Would it make more sense to move it to
>      >         core/include/numpy/private.
>      >          >>
>      >          >> No, the whole point is to avoid other packages to include
>      >         that by
>      >          >> mistake, to avoid namespace pollution.
>      >          >
>      >          > Isn't that what the npy prefix is for?
>      >
>      >         No, npy_ is for public symbols. Anything in private should be
>      >         private :)
>      >
>      >          > In any case, if it needs to be at a
>      >          > higher level for easy inclusion, then it should move up.
>      >
>      >         It is not that easy - we should avoid putting this code into
>      >         core/include, because then we have to keep it compatible
>     across
>      >         releases, but there is no easy way to share headers
>     between modules
>      >         without making it public.
>      >
>      >
>      >     Py_TYPE, Py_Size, etc. are unlikely to cause compatibility
>     problems
>      >     across releases.
>      >
>      >
>      >
>      > In particular, I think
>      >
>      > #if (PY_VERSION_HEX < 0x02060000)
>      > #define Py_TYPE(o)    (((PyObject*)(o))->ob_type)
>      > #define Py_REFCNT(o)  (((PyObject*)(o))->ob_refcnt)
>      > #define Py_SIZE(o)    (((PyVarObject*)(o))->ob_size)
>      > #endif
>      >
>      > belongs somewhere near the top, maybe with a prefix (cython seems to
>      > define them also)
> 
>     The rule is easy: one should put in core/include/numpy whatever is
>     public, and put in private what is not.
> 
>     Note that defining those macros above publicly is very likely to cause
>     trouble because I am sure other people do define those macros, without
>     caring about polluting the namespace as well. Given that it is
>     temporary, and is small, I think copying the compat header is better
>     than making it public, the best solution being to add something in
>     distutils to share it between submodules,
> 
> 
> You would prefer to fix the macros in ndarrayobject.h using #ifdef's then?

In case what I am worried about is not clear: if ndarrayobject.h defines 
Py_TYPE, it means that every C extensions using the numpy C API will 
have Py_TYPE in the public namespace. Now, if another python extension 
with a C API does the same, you have issues. Having #ifdef/#endif around 
only make it worse because then you have strange interactions depending 
on the order of header inclusion (I really hate that behavior from the 
python headers).

The numpy C headers are already pretty messy, let's not make it worse. 
Especially since the workaround is trivial.

cheers,

David