M.-A. Lemburg wrote:

On 2009-03-03 10:38, Stefan Behnel wrote:

...

Hrvoje Niksic wrote:

...

Stefan Behnel writes:

...

pycnweb wrote:

...

I am now writing a python extension, but I confuse with the reference. If you care more about working code than fighting confusion, Cython might be worth a look. Stefan, please stop doing this. This list is about helping people using Python/C, not Cython advocacy.

Where is the difference?

The list would be called cython-sig otherwise :-)

There are many other alternatives to approaching Python extension writing in C/C++:

• use ctypes
• use Boost.Python
• use SWIG
• use sip
• use PyCXX
• use PyRex/Cython
• use Robin
• ...

(there are lots more)

I know almost all of these at least by name. However, most of them are wrapper generators, which clearly limits their field of applicability (and rarely leads to good wrappers without major manual interaction). Cython is not a wrapper generator. It's a programming language that sits right between Python and C and generates code for the C-API of Python. That's why its related to this mailing list and why I advocate it to people who are clearly starting to write code against that API. It just makes their life a lot easier without sacrificing anything.

It's similar to what the Jython developers say, "we write C so you don't have to". If you ever need to make a direct call to Python's C-API in Cython code, it's one cimport away. In all other cases, it's better to concentrate on real logic and algorithmic optimisations in working code, than on counting references, converting types, building dicts and writing extension class structs. It's pretty close to Python's own spirit in that regard. Plus, no matter how long you grow your code, Cython just doesn't start to get in your way - which has always been the case for any wrapper generator I've ever worked with.

I do see ctypes as a pure-Python-level alternative, but it's not related to this list in any obvious way.

However, this list is about the layer underneath all of those. It's about the Python C API and there are many good reasons to use it directly rather than using one of the above wrappers or wrapper generators.

I agree regarding the wrapper generators, but I do not agree regarding the need to call Python's C-API. I know that from personal experience as the author of lxml (which is written in Cython). lxml's code uses direct C-API calls in quite a few places, but that's almost purely for historical reasons. Today, most of that has become a premature performance optimisation that only makes the code less obvious. Last year, I started refactoring that back into Python/Cython code, because in 95% of the code, there just isn't any reason left to use direct C-API calls. Plain Cython code is at least as fast and often faster than hand-written code, especially when it comes to interaction with the Python runtime.

Call me a burnt child, I hope this makes it understandable that I'm trying to keep new C-API users from investing tons of time into their C code, just to notice that it will finally run, say, 30% slower and be less portable than what they could have had with 10-15% of the code size in Cython. And since the factor of 10 in code size is very real, it leaves you with 80-95% of the plain-C development time to benchmark and optimise your code.

Stefan