[Twisted-Python] Possibly of interest
[ Please excuse me if this isn't of interest. I saw this and suspected that people here might be interested (whether agreed with or not) though - if it isn't, my apologies! ] Hi, I was forwarded a link to this paper just recently (and skimming the past 5-6 months of archives here it doesn't look like it's been posted to this list). Since it looks interesting I thought I'd forward a copy to yourselves. http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_ht... Whilst the paper is a couple of years old, the comments are probably interesting because whilst it describes itself really as "threads are better than events" their implementation is based on C-Coroutine library (that doesn't use threads...) and suggests in its conclusions: "In the future, we advocate tight integration between the compiler and the thread system, which will result in a programming model that offers a clean and simple interface to the programmer while achieving superior performance." Which to me smacks of generators - which IMO can be viewed as just a different way of wrapping up the event stack. (Which doesn't entirely surprise me - one of their influences listed in para 3 of the introduction was Inktomi's Traffic Server - somewhere I worked before the BBC, and came to the same conclusions they did. //Wrongly or rightly//, dunno :-) (On an unrelated note: Tommi - after the discussions we had at Europython I came back and discussed general timelines for making our Kamaelia stuff work transparently with Twisted for an alpha stage. We came to the conclusion that we'd like to push that out to a 1.0.0 timeframe, which would probably be around a year from now. It gives us still the freedom in that respect to be able to answer "use twisted" when people ask us what they should use when using python for production servers :-) Also it means that /if/ we do achieve clean integration, that the answer won't change ;-) Anyway, hopefully the paper is of interest (even if disagreed with!), if it isn't, my apologies! Best Regards, -- Michael Sparks, Senior R&D Engineer, Digital Media Group Michael.Sparks@rd.bbc.co.uk, http://kamaelia.sourceforge.net/ British Broadcasting Corporation, Research and Development Kingswood Warren, Surrey KT20 6NP This e-mail may contain personal views which are not the views of the BBC.
On Mon, 2005-08-22 at 13:23 +0100, Michael Sparks wrote:
I think that stackless (or greenlets) are closest to what they are talking about in the Python domain. Because of the title of their paper, people who like kernel threads keep quoting this paper as evidence (e.g. the C10K page), without bothering to read it and noticing it's not about kernel threads. This is annoying :) As a counterargument to their claim one could point to growing need to deal with multi-CPU machines, something their architecture doesn't address. Also IIRC the gains in performance they got wouldn't necessarily show up in something as slow as Python :) But I read that paper a long time ago so maybe I'm misremembering or they address these issues.
On 8/22/05, Michael Sparks <ms@cerenity.org> wrote:
you have to take this paper in CONTEXT of the previous paper tha one of these same authors wrote about a year earlier. The SEDA paper -> http://www.google.com/url?sa=t&ct=res&cd=5&url=http%3A//www.eecs.harvard.edu/%7Emdw/papers/seda-sosp01.pdf&ei=IQcKQ6ilLMPc4QGu0rGXDg Where they come to the exact OPPOSITE conclusion :-) It is all good reading! They are NOT promoting threads as much as they are promoting a better scheduler for threads. Which is what SEDA ( and Twisted ) provide with the event driven model. Their basic conclusion is that EXISTING threading implementations are too general/generic and a more specialized scheduler that is way more effiecent would make Threaded programs scale better AND have the more friendly Thread style interface. Until someone picks up thair Carpaccio project and make it produciton worthy, we will probably just have to keep dealing with the complexity of event driven async programming idioms like Twisted. -- If you don't know what you want, you probably need a nap.
On Mon, 2005-08-22 at 13:23 +0100, Michael Sparks wrote:
I think that stackless (or greenlets) are closest to what they are talking about in the Python domain. Because of the title of their paper, people who like kernel threads keep quoting this paper as evidence (e.g. the C10K page), without bothering to read it and noticing it's not about kernel threads. This is annoying :) As a counterargument to their claim one could point to growing need to deal with multi-CPU machines, something their architecture doesn't address. Also IIRC the gains in performance they got wouldn't necessarily show up in something as slow as Python :) But I read that paper a long time ago so maybe I'm misremembering or they address these issues.
On 8/22/05, Michael Sparks <ms@cerenity.org> wrote:
you have to take this paper in CONTEXT of the previous paper tha one of these same authors wrote about a year earlier. The SEDA paper -> http://www.google.com/url?sa=t&ct=res&cd=5&url=http%3A//www.eecs.harvard.edu/%7Emdw/papers/seda-sosp01.pdf&ei=IQcKQ6ilLMPc4QGu0rGXDg Where they come to the exact OPPOSITE conclusion :-) It is all good reading! They are NOT promoting threads as much as they are promoting a better scheduler for threads. Which is what SEDA ( and Twisted ) provide with the event driven model. Their basic conclusion is that EXISTING threading implementations are too general/generic and a more specialized scheduler that is way more effiecent would make Threaded programs scale better AND have the more friendly Thread style interface. Until someone picks up thair Carpaccio project and make it produciton worthy, we will probably just have to keep dealing with the complexity of event driven async programming idioms like Twisted. -- If you don't know what you want, you probably need a nap.
participants (3)
-
Itamar Shtull-Trauring -
jarrod roberson -
Michael Sparks