Fake threads (was [Python-Dev] ActiveState & fork & Perl)

Christian Tismer tismer@appliedbiometrics.com
Thu, 08 Jul 1999 16:23:11 +0200

Tim Peters wrote:

> This is a particularly simple application of coroutines that could be
> packaged up in a simpler way for its own sake; so, again, while
> continuations may be used fruitfully under the covers here, there's still no
> reason to make a poor end user wrestle with them.


def longcomputation(prog, *args, **kw):
    return quickreturn(prog, args, kw)

# prog must be something with return function first arg
# quickreturn could be done as so:

def quickreturn(prog, args, kw):
    cont = getpcc() # get parent's continuation
    def jumpback(val=None, cont=cont):
        putcc(cont, val) # jump to continuation
    apply(prog, jumpback, args, kw)

# and if they want to jump out, they call jumpback with 
# an optional return value.

Can't help it, it still is continuation-ish.

> > ... Well, I admit that the continuation approach is slightly too much
> > for the coroutine/generator case,
> It's good that you admit that, because generators alone could have been
> implemented with a 20-line patch <wink>.  BTW, I expect that by far the bulk
> of your changes *still* amount to what's needed for disentangling the C
> stack, right?  The continuation implementation has been subtle, but so far
> I've gotten the impression that it requires little code beyond that required
> for stacklessness.

Right. You will see soon. The only bit which cont's need more than
coro's is to save more than one stack state for a frame.
So, basically, it is just the frame copy operation.
If I was heading just for coroutines, then I could save
that, but then I need to handle special cases like exception,
what to do on return, and so on.
Easier to do that one stuff once right. Then I will never dump
code for an unforeseen coro-effect, since with cont's, I *may*
jump in and bail out wherever I want or don't want.
The special cases come later and will be optimized, and naturally
they will reduce themselves to what's needed.

Example: If I just want to switch to a different coro, I just
have to swap two frames. This leads to a data structure which
can hold a frame and exchange it with another one.
The cont-implementation does something like

fetch my current continuation  # and this does the frame copy stuff
save into local state variable
fetch cont from other coro's local state variable
jump to new cont

Now, if the source and target frames are guaranteed to
be different, and if the source frame has no dormant
extra cont attached, then it is safe to merge the above
steps into one operation, without the need to save
local state.

In the end, two coro's will jump to each other by doing
nothing more than this. Exactly that is what Sam's
prototype does right now. WHat he's missing is treatment
of the return case. If a coro returns towards the place
where it was forked off, then we want to have a cont
which is able to handle it properly.
That's why exceptions work fine with my stuff: You can
put one exceptionhandler on top of all your coroutines
which you create. It works without special knowledge
of coroutines.
After I realized that, I knew the way to go.

> > ...
> > How about "amb"? :-)
> > (see "teach youself schem in fixnum days, chapter 14 at
> > http://www.cs.rice.edu/~dorai/t-y-scheme/t-y-scheme-Z-H-15.html#%_chap_14)
> That's the point at which I think continuations get insane:  it's an
> unreasonably convoluted implementation of a straightforward (via other
> means) backtracking framework.  In a similar vein, I've read 100 times that
> continuations can be used to implement a notion of (fake) threads, but
> haven't actually seen an implementation that wasn't depressingly subtle &
> long-winded despite being just a feeble "proof of concept".

Maybe this is a convoluted implementation. But the principle?
Return a value to your caller, but stay able to continue
and do this again.
Two continuations, and with the optimizations from above,
it will be nothing.

I will show you the code in a few, and you will realize that we are
discussing the empty set. The frames have to be used, and the
frames are already continuations. Only if they can be reached
twice, they will have to be armed for that.

Moving back to my new "more code - less words" principle.

[mutable ints as loop counters]

> Ah, very clever!  Yes, that will fly -- the continuations will share a
> reference to the value rather than the value itself.  Perfect!

Actually I'm copying some code out of Marc's counterobject
which is nothing more than a mutable integer and hide it
in ceval.c, since that doesn't introduce another module
for a thing which isn't needed elsewhere, after Guido's hint.

Better than to use the array module which doesn't publish 
its internals and might not always be linked in.

> Right, no downside at all.  Except that Guido will hate it <wink>.

I made sure that this is what he hates the lest.

off-for-coding-ly y'rs - chris

Christian Tismer             :^)   <mailto:tismer@appliedbiometrics.com>
Applied Biometrics GmbH      :     Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101   :    *Starship* http://starship.python.net
10553 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     we're tired of banana software - shipped green, ripens at home