On Sat, Feb 11, 2012 at 1:40 PM, Matt Joiner <anacrolix@gmail.com> wrote:
I'm asking if it'd actually be accepted in 3.
Why is that relevant? If free threading is the all-singing all dancing wonderment you believe: 1. Fork CPython 2. Make it free-threaded (while retaining backwards compatibility with all the C extensions out there!) 3. Watch the developer hordes flock to your door (after all, it's the lack of free-threading that has held Python's growth back for the last two decades, so everyone will switch in a heartbeat the second you, or anyone else, publishes a free-threaded alternative where all their C extensions work. Right?).
I know well, and have seen how quickly things are blocked and rejected in core (dabeaz and shannon's attempts come to mind). I'm well familiar with previous attempts.
If that's what you think happened, then no, you're not familiar with them at all. python-dev has just a few simple rules for accepting a free-threading patch: 1. All current third party C extension modules must continue to work (ask the folks working on Ironclad for IronPython and cpyext for PyPy how much fun *that* requirement is) 2. Calls to builtin functions and methods should remain atomic (the Jython experience can help a lot here) 3. The performance impact on single threaded scripts must be minimal (which basically means eliding all the locks in single-threaded mode the way CPython currently does with the GIL, but then creating those elided locks in the correct state when Python's threading support gets initialised) That's it, that's basically all the criteria we have for accepting a free-threading patch. However, while most people are quite happy to say "Hey, someone should make CPython free-threaded!", they're suddenly far less interested when faced with the task of implementing it *themselves* while preserving backwards compatibility (and if you think the Python 2 -> Python 3 transition is rough going, you are definitely *not* prepared for the enormity of the task of trying to move the entire C extension ecosystem away from the refcounting APIs. The refcounting C API compatibility requirement is *not* optional if you want a free-threaded CPython to be the least bit interesting in real world terms). When we can't even get enough volunteers willing to contribute back their fixes and workarounds for the known flaws in multiprocessing, do people think there is some magical concurrency fairy that will sprinkle free threading pixie dust over CPython and the GIL will be gone? Removing the GIL *won't* be fun. Just like fixing multiprocessing, or making improvements to the GIL itself, it will be a long, tedious grind dealing with subtleties of the locking and threading implementations on Windows, Linux, Mac OS X, *BSD, Solaris and various other platforms where CPython is supported (or at least runs). For extra fun, try to avoid breaking the world for CPython variants on platforms that don't even *have* threading (e.g. PyMite). And your reward for all that effort? A CPython with better support for what is arguably one of the *worst* approaches to concurrency that computer science has ever invented. If a fraction of the energy that people put into asking for free threading was instead put into asking "how can we make inter-process communication better?", we'd no doubt have a good shared object implementation in the mmap module by now (and someone might have actually responded to Jesse's request for a new multiprocessing maintainer when circumstances forced him to step down). But no, this is the internet: it's much easier to incessantly berate python-dev for pointing out that free threading would be extraordinarily hard to implement correctly and isn't the panacea that many folks seem to think it is than it is to go *do* something that's more likely to offer a good return on the time investment required. My own personal wishlist for Python's concurrency support? * I want to see mmap2 up on PyPI, with someone working on fast shared object IPC that can then be incorporated into the stdlib's mmap module * I want to see multiprocessing2 on PyPI, with someone working on the long list of multiprocessing bugs on the python.org bug tracker (including adding support for Windows-style, non-fork based child processes on POSIX platforms) * I want to see progress on PEP 3153, so that some day we can have a "Python event loop" instead of a variety of framework specific event loops, as well as solid cross-platform async IO support in the stdlib. As Jesse said earlier, asking for free threading in CPython is like asking for free pie. Sure, free pie would be nice, but who's going to bake it? And what else could those people be doing with their time if they weren't busy baking pie? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia