[Python-Dev] Re: uthread strawman

Christian Tismer tismer@tismer.com
Tue, 07 Nov 2000 21:54:14 +0200

Just answering/clarifying a few bits,
since I can't change your opinion about
continuations, anyway.

Guido van Rossum wrote:
> I've thought about it a little more, and drawn some pictures in my
> head.

I have to agree with Guido when he says:

> I still have to disagree with Christian when he says:
> > Making Python completely coroutine aware, without
> > tricking the C stack, is 90 percent of the problem.
> > But after walking that far, there is no reason
> > to leave the other 10 percent alone.

... since I meant implementability. Of course there are
other reasons gainst continuations. I just did it since they
were in reach.

> Without continuations, but with microthreads (uthreads) or coroutines,
> each (Python) stack frame can simply be "paused" at a specific point
> and continued later.  The semantics here are completely clear (except
> perhaps for end cases such as unhandled exceptions and intervening C
> stack frames).

I agree. But also with continuations, the situation is identical,
as long as you don't try anything else where continuations
would be needed.
Note that they will not need to be created when the mentioned
structures are implemented well. We don't have to implement them,
but providing support for them in the interpreter framework
is simple. (that's the 10% issue).

> With continuations, you have to decide how much state to save for a
> future continuation.  It would seem easy enough: save all state kept
> in the frame except for the local variables.  But now consider this:
> the "evaluation stack" contained in each frame could easily be
> replaced by a bunch of temporary variables, if we had a slightly
> different instruction set (3-address opcodes instead of stack-based
> opcodes).  Then would we save those temporary variables or not?  it
> can make a difference!  Since the "save continuation" operation is a
> function call, you can easily save a continuation while there are some
> items on the value stack.  I believe the current implementation saves
> these so they are restored when you jump to the continuation.  But if
> the compiler were to use temporary variables instead of the evaluation
> stack, they might not have been restored!

I would consider these temporary variables registers which must
be preserved. They are immutable objects as part of the immutable
continuation, treated as values. Stack or registers, this is part
of an expression evaluation. Temporary results must conceptually
be read-only, whatever way I implement this.

> Here's another example.  Suppose you set up a for loop.  After three
> iterations through the loop you save a continuation.  Then you finish
> hree more iterations.  Then you return to the saved continuation.
> Where does the loop continue: at 3 or at 6 iterations?  Try to answer
> this without trying it.  My guess: it gets restarted at 3 iterations,
> because the loop index is saved on the value stack.  If you rewrite
> this using a while loop, however, it would get restarted at 6
> iterations, because then your loop index is an unsaved local variable.
> Ditto if you changed the bytecode compiler so for loops use an
> anonymous local variable instead of an entry on the evaluation
> stack.

Wrong guess!
Exactly for that reason I changed the loop code to put a mutable
loopcounter object on the stack.
The loop works perfectly.

> This semantic imprecision is one of the reasons why I don't like the
> concept of continuations.  (I've been told that the exact semantics of
> continuations in Scheme differ greatly between Scheme implementations.)

In a sense, you have continuations already, also with the restriction
to gen/co/uthread structures. The only difference is to treat a
frame as exactly one continuation and to disallow to have more
than one at any time.
This saves the decision about the ambiguities you mentioned.

I agree that going to this point and not further for the
Python core is a good idea.
A PEP doesn't need to name continuations at all.

On the other hand, I don't see a reason why this hsould mean
that Python *must not* support them. What I'd like to keep
is the possibility to still write such an extension module.

Enabling this for educational purposes is a great value
that comes at a cheap price and no impact for the core.

> Now let's look at Jython.  In Jython, we can simulate "paused frames"
> well enough by using real Java threads.  However full continuations
> would require modifications to the JVM -- which is unacceptable to a
> language boasting "100% Pure Java".  Another reason against allowing
> continuations.

Not quite true, after I heard of a paper that shows how
to implement continuations in Java, using threads.
But I'll come back to that when I have the paper.

> So, all in all, I don't think of continuations as "the last 10% that
> we might as well add to finish the job."  I see it as an ill-specified
> hypergeneralization.

Can we agree to not support them without forbidding them?

> A strawman proposal:

Ok, this looks all very well to me. More on that later.
One question: Why do you want an explicit u.yield() ?
Uthreads are scheduled automatically now, like real
threads. Do you see a major drawback in supporting
this, maybe as an option? Or do you see automatic
scheduling as an extra construct on top with a special
"scheduler" uthread?

cheers - chris

Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com