[Python-ideas] Cofunctions - Back to Basics

Matt Joiner anacrolix at gmail.com
Mon Oct 31 01:06:08 CET 2011


This is excellent. A lot of languages are opening up new methods for
concurrency, Python should make sure to participate on this.

In particular Haskell's lightweight threads, "sparks", and golang's
"goroutines" are of this form and also provide builtin asynchronous
IO.

I think a feature like this requires some standardization (in the form
of the standard library) in order that all third-parties can cooperate
on the same implementation.

I'm not sure that option 2 that Nick provides plays nice with the C
compatibility of the CPython implementation. I've had a lot of success
with the greenlet model, it's quite trivial to wrap it up to
implicitly spawn an IO loop under the covers. The downside is that all
the client code needs to be adjusted to defer blocking calls to the
loop, or the entire standard library must be hooked. Again, this
doesn't play well with the C stuff: Native C modules, or any third
party calls that don't expect to be part of the greenlet model will
block entire threads.

I'd love to see suggestions that enable existing C code to function as
expected, otherwise an opt-in system (which is how my own
implementations operate). Again, if some coroutine stuff was baked
into the standard library, that would enable third-parties to reliably
write modules that could rely on support for coroutines being
available.

I find Greg's coroutine PEP confusing, and I don't see why an existing
reliable model can't be used (looking at you greenlets).

On Mon, Oct 31, 2011 at 9:40 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Mon, Oct 31, 2011 at 8:12 AM, Matt Joiner <anacrolix at gmail.com> wrote:
>> +10 for greenlet style coroutines. It's a very modern feature and will put
>> python in an excellent position. Also nick your points about asynchronous io
>> being the main use case are spot on as far as I'm concerned.
>
> This whole thread has been very helpful to me in understanding not
> only why I think Greg's coroutine PEP is important, but also why the
> whole generators-as-coroutines paradigm feels so restrictive.
>
> It means we basically have two potential paths forward:
>
> 1. Stackless Python (aka greenlets).
>   Pros: known to work for a large number of use cases
>   Cons: need to define a mechanism to declare that the C stack is
> being used in a "coroutine friendly" fashion
>
> 2. Implicit creation of generator-style suspendable frames
>   Pros: shouldn't need assembly level hackery
>   Cons: effectively requires duplication of every C level type slot
> with a coroutine friendly equivalent that supports suspension
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>



More information about the Python-ideas mailing list