p
On 03/03/2016 09:58 AM, Eric Snow wrote:
...what would we need to do to our C API to make it so that anyone following a new API wouldn't be broken if we dropped the GIL? If I recall correctly, this was one key topic that Larry discussed at
On Thu, Mar 3, 2016 at 10:39 AM, Brett Cannon brett@python.org wrote: the language summit latest year.
Kinda, yeah. Certainly it's a topic I've thought a lot about.
Consider this. With almost no exceptions*, none of the popular new languages have a C API. Instead they'll have a foreign function interface allowing you to call C from inside the language. So you don't write extensions in C and call into the language, you write your extensions natively in the language and call out.
One advantage of this technique is that it allows most implementation details of the language to remain hidden. CPython can't drop reference counting and move solely to tracing garbage collection, because the C API lets external callers deal with Python objects, which means Python's internal object lifetime management approach must be visible, which means it's implicitly part of the API. In short, if we change from reference counting to tracing GC, we break every C extension in existence, kablooey, oblivion.
But! If there were no C extensions--if all Python programs talked to
native libraries through FFIs like ctypes and cffi--then this would be a
private implementation detail and we could iterate however we liked on
object lifetime management. I've asked Armin Rigo about PyPy here.
Pardon me if my memory is faulty, but what I think he said was this:
they started with GC, then went to generational GC, then went to
incremental generational GC. If they'd had a C API, going to
generational probably wouldn't have broken all their extensions, but
going to incremental absolutely would have. Since PyPy doesn't have a C
API, naturally they can change it all they like.
If we could wave a magic wand and get all extension authors to switch to writing their extensions in Python and using cffi, we should absolutely do it. That'd be great for cross-implementation compatibility; your extension would (hopefully) run unchanged in CPython and PyPy today, and I heard a rumor that Jython and IronPython want to support cffi too, so hey! someday it might run unchanged in those too. This would also make it possible for CPython to declare that the C API was dead and free us up to make some radical but welcome changes to CPython's innards. Unfortunately, we don't have such a magic wand, and I don't think there's any workable path to convince extension authors to switch en masse. And if we're stuck with the C API, we're stuck with a lot of the implementation details that are baked into it.
I'm hoping to present on this subject at this year's summit. I hope all the interested core devs can make it!
//arry/
- The only exception I know of is Lua--are there more?