Fake threads (was [Python-Dev] ActiveState & fork & Perl)

[Christian Tismer]

Guido van Rossum wrote:
...
> I'm trying to understand whether we can get away with saving just a
> pointer to a frame, whether we need to copy the frame, or whether we
> need to copy the entire frame stack.

You need to preserve the stack and the block stack of a frame, if
and only if it can be reached twice. I make this dependent from
its refcount. Every frame monitors itself before and after
every call_function, if a handler field in the frame "f_callguard"
has been set. If so, the callguard is called. Its task is to
see wether we must preserve the current state of the frame and to
carry this out.

The idea is to create a shadow frame "on demand". When I touch
a frame with a refcount > 1, I duplicate it at its f_back pointer.
By that is is turned into a "continuation frame" which is nothing
more than the stack copy, IP, and the block stack.

By that, the frame stays in place where it was, all pointers
are still fine. The "real" one is now in the back, and the
continuation frame's purpose when called is only to restore
the state of the "real one" and run it (after doing a new
save if necessary).

I call this technique "push back frames".

> 
> (In regular Python, the frame stack also contains local variables.
> These are explicitly exempted from being saved by a continuation.  I
> don't know how Christian does this, but I presume he uses the
> dictionary which can be shared between frames.)

I keep the block stack and a stack copy. All the locals are
only existing once. The frame is also only one frame. Actually
always a new one (due to push back), but virtually it is
"the frame", with multiple continuation frames pointing at it.

...
> Clearly if that saved continuation is ever invoked (called?  used?
> activated?  What do you call what you do to a continuation?)

I think of throwing. Mine are thrown. The executive of standard
frames is "eval_code2_loop(f, passed_retval)", where the executive
of a continuation frame is "throw_continuation(f, passed_retval)".

...
> Is it safe not to save the frame and block stacks of frames further
> down on the call stack?  I don't think so -- these are all destroyed
> when frames are popped off the call stack (even if the frame is kept
> alive, its value and block stack are always empty when the function
> has returned).
> 
> So I hope that Christian has code that saves the frame and block
> stacks!  (It would be fun to try and optimize this by doing it lazily,
> so that frames which haven't returned yet aren't copied yet.)

:-) I have exactly that, and I do it lazily already.
Unless somebody saves a continuation, nothing special happens.
But if he does, the push back process follows his path like
a zip (? Reißverschluß) and ensures that the path can be walked
again. Tarzan has now the end of this liane in his hand.
He might use it to swing over, or he might drop it, and it ribbles
away and vanishes as if it never existed.

Give me some final testing, and you will be able to try it out
in a few days.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@appliedbiometrics.com>
Applied Biometrics GmbH      :     Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101   :    *Starship* http://starship.python.net
10553 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     we're tired of banana software - shipped green, ripens at home