[Python-Dev] Let's change to C API!

[Antoine Pitrou]

On Tue, 31 Jul 2018 15:34:05 +0200
Victor Stinner <vstinner at redhat.com> wrote:
> Antoine: would you mind to subscribe to the capi-sig mailing list? As
> expected, they are many interesting points discussed here, but I would
> like to move all C API discussions to capi-sig. I only continue on
> python-dev since you started here (and ignored my request to start
> discussing my idea on capi-sig :-)).

Well, I responded to your e-mail discussion thread.  I see more
messages in this thread here than on capi-sig. ;-)

> For example, PyPy uses different memory allocators depending on the
> scope and the lifetime of an object. I'm not sure that you can
> implement such optimization if you are stuck with reference counting.

But what does reference counting have to do with memory allocators
exactly?

> > OS vendors seem to be doing a fine job AFAICT.  And if I want a recent
> > Python I just download Miniconda/Anaconda.  
> 
> Is it used in production to deploy services? Or is it more used by
> developers? I never used Anaconda.

I don't know, but there's no hard reason why you couldn't use it to
deploy services (though some people may prefer Docker or other
technologies).

> > I think you don't realize that the C API is *already* annoying.  People
> > started with it mostly because there wasn't a better alternative at the
> > time.  You don't need to make it more annoying than it already is ;-)
> >
> > Replacing existing C extensions with something else is entirely a
> > developer time/effort problem, not an attractivity problem.  And I'm
> > not sure that porting a C extension to a new C API is more reasonable
> > than porting to Cython entirely.  
> 
> Do you think that it's doable to port numpy to Cython? It's made of
> 255K lines of C code.

Numpy is a bit special as it exposes its own C API, so porting it
entirely to Cython would be difficult (how do you expose a C macro in
Cython?).  Also, internally it has a lot of macro-generated code for
specialized loop implementations (metaprogramming in C :-)).

I suppose some bits could be (re)written in Cython.  Actually, the
numpy.random module is already a Cython module.

> > It's just that I disagree that removing the C API will make CPython 2x
> > faster.  
> 
> How can we make CPython 2x faster? Why everybody, except of PyPy,
> failed to do that?

Because PyPy spent years working full time on a JIT compiler.  It's also
written in (a dialect of) Python, which helps a lot with experimenting
and building abstractions, compared to C or even C++.

Regards

Antoine.