[Python-Dev] Pythonic concurrency
radeex at gmail.com
Tue Oct 11 03:01:03 CEST 2005
On 10/11/05, Guido van Rossum <guido at python.org> wrote:
> I recall using a non-preemptive system in the past; in Amoeba, to be precise.
> Initially it worked great.
> But as we added more powerful APIs to the library, we started to run
> into bugs that were just as if you had preemptive scheduling: it
> wouldn't always be predictable whether a call into the library would
> need to do I/O or not (it might use some sort of cache) so it would
> sometimes allow other threads to run and sometimes not. Or a change to
> the library would change this behavior (making a call that didn't use
> to block into sometimes-blocking).
I'm going to be giving a talk at OSDC (in Melbourne) this year about
concurrency systems, and I'm going to talk a lot about the subtleties
between these various non-preemptive (let's call them cooperative :)
systems. I advocate a system that gives you really
straightforward-looking code, but still requires you to annotate the
fact that context switches can occur on every frame where they might
occur (i.e., with a yield). I've given examples before of my new
2.5-yield + twisted Deferred code here, but to recap it just means
that you have to do:
x = yield getPage()
when you want to download a web page, and the caller of 'foo' would
*also* need to do something like "yay = yield foo()". I think this is
a very worthwhile tradeoff for those obsessed with "natural" code.
Twisted | Christopher Armstrong: International Man of Twistery
Radix | -- http://radix.twistedmatrix.com
| Release Manager, Twisted Project
\\\V/// | -- http://twistedmatrix.com
|o O| |
More information about the Python-Dev