[Python-Dev] Stackless Python
Phillip J. Eby
pje at telecommunity.com
Tue Jun 1 12:21:00 EDT 2004
At 06:34 AM 6/1/04 -0700, Guido van Rossum wrote:
> > It is most probably possible to implement soft-switchign in Jython.
>That's a rather vague statement; I'd like to hear from Samuele about
Me too. The only thing I've heard of in Java that does lightweight
co-operative task switching, is actually implemented using a Java VM
written in Java. Sort of like PyPy, only I guess you'd call it
JavaJava. :) I don't remember the name of it right off, just that it was
a commercial product that claimed they had a patent pending.
> > No idea what you mean by arbitrary exceptions (I never had that)
>I meant exceptions to the general rules. Like, you can't depend on
>tasklet switching when you use one of these extensions, or one of
>these built-ins, or ...
Perhaps it would be better if it were defined in terms of where it *does*
work, rather than where it doesn't. For example, one can write
co-operative multitasking code in CPython now using generators... but it
only works if you have "generators all the way down". That is, any code in
a task that calls a generator must itself be a generator. That's a pretty
comprehensible rule, albeit draconian to follow. Adding soft-switching
would change the rule to be, "you can switch as long as you haven't been
called from C code." That is, you haven't been called from *any* built-in
or extension module.
Speaking as someone who's written some significant event-driven
co-operative multitasking code using generators, I would say that going
from "generators all the way down" to "Python all the way down" would be a
huge improvement in flexibility.
> > On the latter: Well, this is a pure interface issue. I don't say
> > that everybody must love tasklets, there are other approaches and
> > interfaces possible. But they all rely on the non-recursive
> > interpreter core, and that's the point, IMHO.
>Is it really just tasklets that the users want?
I am not currently a Stackless user, but if I were to try it, tasklets
would be my primary interest. I don't actually think that the higher
recursion limit is a "feature", for two reasons: 1) Python loops run a lot
faster than function calls, and 2) every time I've ever gotten an error
from excessive recursion, it was because I'd accidentally created an
infinite recursion. So, to me, lifting the default recursion limit in
standard Python would be a really *bad* idea.
The other feature besides tasklets that I find potentially interesting is
thread pickling. I'm intrigued by the possibility of using them to create
"long-running tasks". That is, workflow using Python scripts to define the
loops, branches, etc. of the workflow. But to make a really practical
workflow system based on this, one would have to have a way of dealing with
changes to the code that the running workflows were based on, so I don't
know if that whole idea is really worth pursuing. I would probably be
better off basing such an animal on PyPy, because performance for the code
in a long-running process workflow is a non-issue.
Anyway, I don't know about "the users" in general, but for me at least
tasklets are where it's at. :) And I only care about explicit yielding,
not thread-like concurrency. (IOW, I don't want Stackless doing the
scheduling, I'd want to be able to write my own scheduler, in pure Python.)
> > I understood it more like the more advanced question "why does
> > Python still block itself from lightweight threading by keping state
> > on the C stack".
>IMO mostly because Python wants to be friendly to extensions written
>in C or C++, including those that frequently call back into Python.
That would only disable task-switching within the called Python code, not
from the code that called the extension. And that, only if the
hard-switching module wasn't available.
More information about the Python-Dev