Confusion with meaning of "Stackless"
Tasklets are nice, but I just wish to update my understanding of "Stackless". Does the old motto "A Python Implementation That Does Not Use The C Stack" [1] still apply? Is it relevant to PyPy? [1] http://stackless.com/index_old.htm Lenard Lindstrom <len-l@telus.net>
Hi Lenard, On Wed, Dec 13, 2006 at 03:50:21PM -0800, Lenard Lindstrom wrote:
Tasklets are nice, but I just wish to update my understanding of "Stackless". Does the old motto "A Python Implementation That Does Not Use The C Stack" [1] still apply? Is it relevant to PyPy?
Yes and no. A bit confusingly, the usage of the term has evolved, along with the Stackless Python project itself. Now, "a Stackless feature" means a feature that cannot be implemented without some form of explicit stack control, either by being very careful about the C code (e.g. avoiding recursive calls) or by saving and restoring pieces of the C stack by a brute force 'memcpy' approach. In PyPy, we use a variant of the former approach: a "stackless build" of pypy-c is a compiled version of some C code that was generated with careful systematic tweaks. The motto of a Stackless PyPy is probably: "A Python implementation that uses the C Stack as a cache". Indeed, the C code supports saving its own stack of frames away into the heap, and restoring it from there, frame by frame. So the C stack is really just a cache for the heap. If the cache is full (stack overflow), we save some more frames into the heap to free some cache space. If a context switch occurs (coroutine switch) the cache is invalidated (entierely flushed to the heap) and the new context's frames are copied from the heap to the cache as needed (the function's frames are resumed one by one). A bientot, Armin.
Armin Rigo <arigo@tunes.org> writes:
Hi Lenard,
On Wed, Dec 13, 2006 at 03:50:21PM -0800, Lenard Lindstrom wrote:
Tasklets are nice, but I just wish to update my understanding of "Stackless". Does the old motto "A Python Implementation That Does Not Use The C Stack" [1] still apply? Is it relevant to PyPy?
Yes and no. A bit confusingly, the usage of the term has evolved, along with the Stackless Python project itself. Now, "a Stackless feature" means a feature that cannot be implemented without some form of explicit stack control, either by being very careful about the C code (e.g. avoiding recursive calls) or by saving and restoring pieces of the C stack by a brute force 'memcpy' approach. In PyPy, we use a variant of the former approach: a "stackless build" of pypy-c is a compiled version of some C code that was generated with careful systematic tweaks.
The motto of a Stackless PyPy is probably: "A Python implementation that uses the C Stack as a cache". Indeed, the C code supports saving its own stack of frames away into the heap, and restoring it from there, frame by frame. So the C stack is really just a cache for the heap. If the cache is full (stack overflow), we save some more frames into the heap to free some cache space. If a context switch occurs (coroutine switch) the cache is invalidated (entierely flushed to the heap) and the new context's frames are copied from the heap to the cache as needed (the function's frames are resumed one by one).
Do you feel like adding this explanation to the D07.1 report? :-) Cheers, mwh -- <radix> A VoIP server "powered entirely by stabbing, that I made out of this gun I had" -- from Twisted.Quotes
On 14 Dec 2006 at 7:48, Armin Rigo wrote:
Hi Lenard,
On Wed, Dec 13, 2006 at 03:50:21PM -0800, Lenard Lindstrom wrote:
Tasklets are nice, but I just wish to update my understanding of "Stackless". Does the old motto "A Python Implementation That Does Not Use The C Stack" [1] still apply? Is it relevant to PyPy?
Yes and no. A bit confusingly, the usage of the term has evolved, along with the Stackless Python project itself. Now, "a Stackless feature" means a feature that cannot be implemented without some form of explicit stack control, either by being very careful about the C code (e.g. avoiding recursive calls) or by saving and restoring pieces of the C stack by a brute force 'memcpy' approach. In PyPy, we use a variant of the former approach: a "stackless build" of pypy-c is a compiled version of some C code that was generated with careful systematic tweaks.
The motto of a Stackless PyPy is probably: "A Python implementation that uses the C Stack as a cache". Indeed, the C code supports saving its own stack of frames away into the heap, and restoring it from there, frame by frame. So the C stack is really just a cache for the heap. If the cache is full (stack overflow), we save some more frames into the heap to free some cache space. If a context switch occurs (coroutine switch) the cache is invalidated (entierely flushed to the heap) and the new context's frames are copied from the heap to the cache as needed (the function's frames are resumed one by one).
Hi Armin, Thanks for the explanation. I find that "Stackless" is still associated with Scheme like continuations and unlimited recursion [1]. But yes, the PyPy documentation clearly states that "Stackless" means greenlets, coroutines and tasklets at the application level. [1] http://en.wikipedia.org/wiki/Stackless Lenard Lindstrom <len-l@telus.net>
Lenard Lindstrom wrote:
On 14 Dec 2006 at 7:48, Armin Rigo wrote:
...
Thanks for the explanation. I find that "Stackless" is still associated with Scheme like continuations and unlimited recursion [1]. But yes, the PyPy documentation clearly states that "Stackless" means greenlets, coroutines and tasklets at the application level.
Well, I should try to update that page. While it is still true, the basic machinery can support continuations, and infinite recursion was available in every Stackless. The "tamed" continuations which are greenlets, coroutines or tasklets seem to be what is much more acceptable for the users. ciao - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 177 402 25 57 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 where do you want to jump today? http://www.stackless.com/
On 18 Dec 2006 at 21:15, Christian Tismer wrote:
Lenard Lindstrom wrote:
On 14 Dec 2006 at 7:48, Armin Rigo wrote:
...
Thanks for the explanation. I find that "Stackless" is still associated with Scheme like continuations and unlimited recursion [1]. But yes, the PyPy documentation clearly states that "Stackless" means greenlets, coroutines and tasklets at the application level.
Well, I should try to update that page. While it is still true, the basic machinery can support continuations, and infinite recursion was available in every Stackless. The "tamed" continuations which are greenlets, coroutines or tasklets seem to be what is much more acceptable for the users.
Taskets interest people. But so did infinite recursion. That it is still a part of Stackless Python is worth mentioning. And if Stackless PyPy supports it then infinite recursion should be documented as a user level feature along with tasklets and such. It does effect the choices a coder makes. As for continuations, I have only seen them advertised as a language feature when the language provides a call/cc or something. Tasklets, coroutines, generators and such are basically treated as independent features. Lenard Lindstrom <len-l@telus.net>
Hi Lenard, On Mon, Dec 18, 2006 at 04:15:48PM -0800, Lenard Lindstrom wrote:
Taskets interest people. But so did infinite recursion. That it is still a part of Stackless Python is worth mentioning. And if Stackless PyPy supports it then infinite recursion should be documented as a user level feature along with tasklets and such. It does effect the choices a coder makes.
Good point. I've added it to stackless.html. A bientot, Armin
participants (4)
-
Armin Rigo -
Christian Tismer -
Lenard Lindstrom -
Michael Hudson