[Numpy-discussion] Notes from the numpy dev meeting at scipy 2015

[Nathaniel Smith]

On Tue, Aug 25, 2015 at 5:53 PM, David Cournapeau <cournape at gmail.com> wrote:
> Thanks for the good summary Nathaniel.
>
> Regarding dtype machinery, I agree casting is the hardest part. Unless the
> code has changed dramatically, this was the main reason why you could not
> make most of the dtypes separate from numpy codebase (I tried to move the
> datetime dtype out of multiarray into a separate C extension some years
> ago). Being able to separate the dtypes from the multiarray module would be
> an obvious way to drive the internal API change.

For practical reasons I don't imagine we'll ever want to actually move
the core dtypes out of multiarray -- if nothing else they will always
remain a little bit special, like np.array([1.0, 2.0]) will just
"know" that this should use the float64 dtype. But yeah, in general a
good heuristic would be that -- aside from a few limited cases like
that -- we want to make built-in dtypes and user-defined dtypes use
the same APIs.

> Regarding the use of cython in numpy, was there any discussion about the
> compilation/size cost of using cython, and talking to the cython team to
> improve this ? Or was that considered acceptable with current cython for
> numpy. I am convinced cleanly separating the low level parts from the python
> C API plumbing would be the single most important thing one could do to make
> the codebase more amenable.

It's still a more blue-sky idea than that... the discussion was more
at the level of "is this something that is even worth trying to make
work and seeing where the problems are?"

The big immediate problem, before we got into code size issues, would
be that we would need to be able to compile a mix of .pyx files and .c
files into a single .so, while cython generated code currently makes
some strong assumptions about how each .pyx file will live in its own
.so. From playing around with it I suspect the first version of making
this work will be klugey indeed. But yeah, the thing to do would be
for someone to dig in and make the kluges and then decide how to clean
them up once you know where they are.

-n

-- 
Nathaniel J. Smith -- http://vorpus.org