Greg Ewing wrote:
M.-A. Lemburg wrote:
Greg Ewing wrote:
In an application that requires thousands of small, cooperating processes,
Sure, and those use Stackless to solve the problem, which IMHO provides a much more Pythonic approach to these things.
At the expense of using a non-standard Python installation, though. I'm trying to design something that can be incorporated into standard Python and work without requiring any deep black magic. Guido has so far rejected any idea of merging Stackless into CPython.
The problem with doing so is twofold:
1. The use case Stackless addresses is not something an everyday programmer will need, so making CPython more complicated just to add this one extra feature, doesn't appear worth the trouble.
2. The Stackless implementation is not very portable, so the feature would only be available on a limited number of platforms.
Apart from that, every new feature will raise the bar for learning Python.
If you could turn your proposal into something more like the Stackless tasklets and move the implementation to an extension module (perhaps with some extra help from new CPython APIs), then I'm sure the proposal would get more followers.
Also I gather that Stackless works by copying pieces of C stack around, which is probably more lightweight than using an OS thread, but not as light as it could be.
Well, it works great in practice and is a proven approach. Copying in C is certainly fast enough for most needs and the black magic is well hidden in Stackless.
And I'm not sure what criteria to judge pythonicity by in all this.
"explicit is better than implicit".
Tasklets are normal Python objects wrapping functions. The create of those tasklets is explicit, not implicit via some (special) yield burried deep in the code.
Stackless tasklets work without requiring any kind of function or call markers -- everything looks exactly like normal Python code.
Right, because everything *is* normal Python code. Tasklets are much more like threads to the programmer, i.e. a well understood concept.
But Guido and others seem to be objecting to my implicit-cocall proposal on the basis that it looks *too much* like normal code. It seems to me that the same criticism should apply even more to Stackless.
I think an important part of the criticism is hiding the fact that you are writing a cofunction away inside the function definition itself.
Generators have the same problem, but at least you can call them as regular Python functions and they only work a little different than normal functions.
The techniques used by Stackless to achieve this are nasty, but then Python also ships with ctypes which relies on similar nasty techniques
But at least it works provided you write your ctypes code correctly and the library you're calling isn't buggy. I seem to remember that there are certain C libraries that break Stackless because they assume that their C stack frames don't move around.
Well, viruses will have a harder time for sure ;-) I am not aware of other use cases that would need to know the location of the stack frame in memory.
BTW: I'm sure that the functionality needed by Stackless could also be moved into a C lib for other languages to use (much like the libffi code).