This is excellent. A lot of languages are opening up new methods for concurrency, Python should make sure to participate on this.
In particular Haskell's lightweight threads, "sparks", and golang's "goroutines" are of this form and also provide builtin asynchronous IO.
I think a feature like this requires some standardization (in the form of the standard library) in order that all third-parties can cooperate on the same implementation.
I'm not sure that option 2 that Nick provides plays nice with the C compatibility of the CPython implementation. I've had a lot of success with the greenlet model, it's quite trivial to wrap it up to implicitly spawn an IO loop under the covers. The downside is that all the client code needs to be adjusted to defer blocking calls to the loop, or the entire standard library must be hooked. Again, this doesn't play well with the C stuff: Native C modules, or any third party calls that don't expect to be part of the greenlet model will block entire threads.
I'd love to see suggestions that enable existing C code to function as expected, otherwise an opt-in system (which is how my own implementations operate). Again, if some coroutine stuff was baked into the standard library, that would enable third-parties to reliably write modules that could rely on support for coroutines being available.
I find Greg's coroutine PEP confusing, and I don't see why an existing reliable model can't be used (looking at you greenlets).
On Mon, Oct 31, 2011 at 9:40 AM, Nick Coghlan ncoghlan@gmail.com wrote:
On Mon, Oct 31, 2011 at 8:12 AM, Matt Joiner anacrolix@gmail.com wrote:
+10 for greenlet style coroutines. It's a very modern feature and will put python in an excellent position. Also nick your points about asynchronous io being the main use case are spot on as far as I'm concerned.
This whole thread has been very helpful to me in understanding not only why I think Greg's coroutine PEP is important, but also why the whole generators-as-coroutines paradigm feels so restrictive.
It means we basically have two potential paths forward:
- Stackless Python (aka greenlets).
Pros: known to work for a large number of use cases Cons: need to define a mechanism to declare that the C stack is being used in a "coroutine friendly" fashion
- Implicit creation of generator-style suspendable frames
Pros: shouldn't need assembly level hackery Cons: effectively requires duplication of every C level type slot with a coroutine friendly equivalent that supports suspension
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia