On Tue, Aug 25, 2015 at 5:53 PM, David Cournapeau <cournape@gmail.com> wrote:
Thanks for the good summary Nathaniel.
Regarding dtype machinery, I agree casting is the hardest part. Unless the code has changed dramatically, this was the main reason why you could not make most of the dtypes separate from numpy codebase (I tried to move the datetime dtype out of multiarray into a separate C extension some years ago). Being able to separate the dtypes from the multiarray module would be an obvious way to drive the internal API change.
For practical reasons I don't imagine we'll ever want to actually move the core dtypes out of multiarray -- if nothing else they will always remain a little bit special, like np.array([1.0, 2.0]) will just "know" that this should use the float64 dtype. But yeah, in general a good heuristic would be that -- aside from a few limited cases like that -- we want to make built-in dtypes and user-defined dtypes use the same APIs.
Regarding the use of cython in numpy, was there any discussion about the compilation/size cost of using cython, and talking to the cython team to improve this ? Or was that considered acceptable with current cython for numpy. I am convinced cleanly separating the low level parts from the python C API plumbing would be the single most important thing one could do to make the codebase more amenable.
It's still a more blue-sky idea than that... the discussion was more at the level of "is this something that is even worth trying to make work and seeing where the problems are?" The big immediate problem, before we got into code size issues, would be that we would need to be able to compile a mix of .pyx files and .c files into a single .so, while cython generated code currently makes some strong assumptions about how each .pyx file will live in its own .so. From playing around with it I suspect the first version of making this work will be klugey indeed. But yeah, the thing to do would be for someone to dig in and make the kluges and then decide how to clean them up once you know where they are. -n -- Nathaniel J. Smith -- http://vorpus.org