How to call and execute C code in Python?

[Stefan Behnel]

Mark Lawrence, 13.05.2012 19:44:
> On 13/05/2012 18:38, Stefan Behnel wrote:
>> Mark Lawrence, 13.05.2012 19:23:
>>> On 13/05/2012 16:39, Chris Angelico wrote:
>>>> On Sun, May 13, 2012 at 11:25 PM, David Shi wrote:
>>>>> Can anyone tell me how to call and exectute C code in Python?
>>>>
>>>> Browse the documentation about Extending and Embedding Python, there's
>>>> an extensive API.
>>>
>>> I like your response, my first thought was to say "yes" :)
>>
>> It has a serious learning curve all by itself, though, with a lot of major
>> pitfalls not only for new users. It also requires C fluency, which not
>> everyone has or wants to learn just to be able to talk to existing C code.
>> And even when you're up to speed with all that, you will waste a
>> substantial part of your time debugging crashes, making your code reference
>> leak free and eventually maintaining it and porting it to different CPython
>> versions and platforms. Time that IMHO is much better spent adding features
>> and tuning the code for performance.
> 
> Haven't the faintest idea what you're on about so please explain.

Sure.

Learning curve and pitfalls: almost all C-API code I've seen from new users
had at least one reference leak, usually more than one. This even happens
to core developers from time to time, so I think it's reasonable to assume
that pretty much all users of the C-API have produced at least one
reference leak in their time as programmers and at least some do so
frequently while learning to use it. I call that a common pitfall.

C fluency: obviously, you need to be able to (and want to) write C if you
want to make effective use of CPython's C-API from C code. Not every Python
developer knows C, and why should they?

Wasting time debugging crashes and making your code reference leak free:
well, if you have ref-counting bugs in your code, you have to find them and
fix them. First of all, you have to know they are there, which isn't always
obvious, especially for leaks. The problem with ref-counting bugs is that
the effects (even crashes) are not local in most cases but can happen
anywhere in your program at unpredictable times, so you will spend more
time on average debugging them than with other, more local bugs.

Wasting time maintaining: arguably, maintaining C code requires more work
than maintaining Python code. That's why we use high-level languages in the
first place, isn't it?

Wasting time porting: well, the C-API has subtle to major differences
between CPython versions, so you not only have to learn "the" C-API, but
the differences between these C-APIs, then write or adapt your code in
order to make it work with different versions. Also, being able to write C
code doesn't mean that that code will compile and run with different C
compilers and on different platforms, so more work on that front.

Spending more time adding features and tuning code instead: well, we
designed Cython to relieve programmers from all of the above, so that they
can focus on functionality and performance. And it's pretty good in that
(you may want to read the testimonials on our project home page).
Basically, the idea is that we write C so you don't have to.

Is it clearer what I mean now?

Stefan