[Numpy-discussion] NumPy re-factoring project

[Sebastien Binet]

On Fri, 11 Jun 2010 00:25:17 +0200, Sturla Molden <sturla at molden.no> wrote:
> Den 10.06.2010 22:07, skrev Travis Oliphant:
> >
> >> 2. The core should be a plain DLL, loadable with ctypes. (I know David Cournapeau and Robert Kern is going to hate this.) But if Python can have a custom loader for .pyd files, so can NumPy for it's core DLL. For ctypes we just need to specify a fully qualified path to the DLL, which can be read from a config file or whatever.
> >>      
> > This approach does not build a new Python type in compiled code.   There are speed disadvantages to this --- especially for the numpy scalars.
> 
> There are at least four speed penalties in what I suggested:
> 
> - the ctypes overhead is bigger than using Python's C API manually (or 
> Cython).
> - there is a speed penalty for scalars and small arrays. (Previously 
> seen in numarray vs. numeric.)
> - bytearray is a bit slower to create than to malloc a buffer.
> - arrays with dtype=object
> 
> The advantages are:
> 
> - the code is easier to read and maintain (clean separation of Python and C)
> - more portable code (no Python C API dependence)
> - no obscure memory leaks to track down (bytearray instead of 
> malloc/free and no manual ref counting).
> 
> 
> By the way: Is Cython mature enough to toss all C out the door? Since 
> Cython has syntax for PEP 3118 buffer objects, we should theoretically 
> be able to implement NumPy in Cython. Then we don't have the speed 
> penalty and no difficult C to maintain. But if the idea is to make a 
> Python independent C library from the core, it is probably a bit counter 
> productive...

as a long time lurker, I really like the idea of having a .so as core
numpy but I'd really hate to have to go thru ctypes as a necessary (or
rather, default) layer to use numpy-core from python.
it of course depends on the granularity at which you wrap and use
numpy-core but tight loops calling ctypes ain't gonna be pretty
performance-wise.

I really think using cython would be a better option.
one'd get the python2 <-> python3 transition for "free".

but, again, I am just a lurker here on numpy-list :)

cheers,
sebastien.

-- 
#########################################
# Dr. Sebastien Binet
# Laboratoire de l'Accelerateur Lineaire
# Universite Paris-Sud XI
# Batiment 200
# 91898 Orsay
#########################################