[Python-ideas] PEP: Hide implementation details in the C API

Nick Coghlan ncoghlan at gmail.com
Tue Jul 11 08:22:50 EDT 2017

On 11 July 2017 at 20:19, Victor Stinner <victor.stinner at gmail.com> wrote:
> Hi,
> This is the first draft of a big (?) project to prepare CPython to be
> able to "modernize" its implementation. Proposed changes should allow
> to make CPython more efficient in the future. The optimizations
> themself are out of the scope of the PEP, but some examples are listed
> to explain why these changes are needed.

Please don't use the word "needed" for speed increases, as we can just
as well diagnose the problem with the status quo as "The transition
from writing and publishing pure Python modules to writing and
publishing pre-compiled accelerated modules in Cython, C, C++, Rust,
or Go is too hard, so folks mistakenly think they need to rewrite
their whole application in something else, rather than just
selectively replacing key pieces of it".

After all, we already *have* an example of a breakout success for
writing Python applications that rival C and FORTRAN applications for
raw speed: Cython.

By contrast, most of the alternatives that have attempted to make
Python faster without forcing users to give up on some the language's
dynamism in the process have been plagued by compatibility challenges
and found themselves needing to ask for the language's runtime
semantics to be relaxed in one way or another.

That said, trying to improve how we manage the distinction between the
public API and the interpreter's internal APIs is still an admirable
goal, and it would be *great* to have CPython natively provide the
public API that cffi relies on (so that other projects could also
effectively target it), so my main request is just to reign in the
dramatic rhetoric and start by exploring how many of the benefits
you'd like can be obtained *without* a hard compatibility break. While
the broad C API is one of CPython's greatest strengths that enabled
the Python ecosystem to become the powerhouse that it is, it is *also*
a pain to maintain consistently, *and* it poses problems for some
technical experiments various folks would like to carry out.

Those kinds of use cases are more than enough to justify changes to
the way we manage our public header files - you don't need to dress it
up in "sky is falling" rhetoric founded in the fear of other
programming languages.

Yes, Python is a nice language to program in, and it's great that we
can get jobs where we can get paid to program in it. That doesn't mean
we have to treat it as an existential threat that we aren't always
going to be the best choice for everything :)


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-ideas mailing list