[Twisted-python] Writing industrial grade twisted servers
Hi all, I've been reading through the twisted documentation, trying to get a handle on how to write a real life twisted app. ( I need to use the AsynQueue for my project, btw..) So I've been reading the documentation, and I've been putting together some things.. but i note that much of the docs says that "this is not the way you usually do this" Where do i read to find out how to simply create a production quality twisted application? -- Kevin Beckford Technical Lead, Lazyweb Construction Company, http://lazyweb.ca kevin@lazyweb.ca
On Tue, 2008-06-17 at 08:49 -0400, kevin beckford wrote:
Where do i read to find out how to simply create a production quality twisted application?
There are some better *deployment* techniques documented here - http://twistedmatrix.com/projects/core/documentation/howto/tap.html - but they just make things easier to deploy, they don't make your software "production quality." Production quality software is hard; a good start is having code reviews, well-written unit tests that cover all your code, as well as end-to-end regression tests. The deployment techniques above do tend to lead to slightly more testable software (by encouraging you to write your software as Services). Here's what we do for Twisted - it has dramatically improved the quality of code we produce: http://twistedmatrix.com/trac/wiki/ReviewProcess
On Jun 17, 2008, at 10:08 AM, Itamar Shtull-Trauring wrote:
On Tue, 2008-06-17 at 08:49 -0400, kevin beckford wrote:
Where do i read to find out how to simply create a production quality twisted application?
There are some better *deployment* techniques documented here - http://twistedmatrix.com/projects/core/documentation/howto/tap.html - but they just make things easier to deploy, they don't make your software "production quality."
Production quality software is hard; a good start is having code reviews, well-written unit tests that cover all your code, as well as end-to-end regression tests. The deployment techniques above do tend to lead to slightly more testable software (by encouraging you to write your software as Services).
Unless I misunderstood the first mail, I don't think this is what Kevin meant . I believe what he is looking for could better be termed "best practice" (it is probably the reference to "industrial grade" in the subject that is misleading). This is something I've struggled with as well with respect to Twisted. Twisted is large and complex and in many cases there are several ways to achieve the same functionality. Frequently, it isn't easy to figure out the "best" approach or even the generally accepted approach. As Kevin says, the documentation does make obscure references to there being some "way you usually do this", without coming out and saying what that way is. All of the code reviews and unit tests in the world aren't going to catch unconventional uses of a library unless one of the reviewers knows better... I'd offer some advice of my own, but I'm not sure even now that I use Twisted properly - at best I have something that works, and as all of my work is experimental that has been good enough.
On Tue, 17 Jun 2008 11:36:40 -0400, Black <python@blackslens.com> wrote:
On Jun 17, 2008, at 10:08 AM, Itamar Shtull-Trauring wrote:
On Tue, 2008-06-17 at 08:49 -0400, kevin beckford wrote:
Where do i read to find out how to simply create a production quality twisted application?
There are some better *deployment* techniques documented here - http://twistedmatrix.com/projects/core/documentation/howto/tap.html - but they just make things easier to deploy, they don't make your software "production quality."
Production quality software is hard; a good start is having code reviews, well-written unit tests that cover all your code, as well as end-to-end regression tests. The deployment techniques above do tend to lead to slightly more testable software (by encouraging you to write your software as Services).
Unless I misunderstood the first mail, I don't think this is what Kevin meant . I believe what he is looking for could better be termed "best practice" (it is probably the reference to "industrial grade" in the subject that is misleading). This is something I've struggled with as well with respect to Twisted. Twisted is large and complex and in many cases there are several ways to achieve the same functionality. Frequently, it isn't easy to figure out the "best" approach or even the generally accepted approach. As Kevin says, the documentation does make obscure references to there being some "way you usually do this", without coming out and saying what that way is. All of the code reviews and unit tests in the world aren't going to catch unconventional uses of a library unless one of the reviewers knows better...
I'd offer some advice of my own, but I'm not sure even now that I use Twisted properly - at best I have something that works, and as all of my work is experimental that has been good enough.
A way to get better answers to this question is to file tickets for each place in the documentation which makes this unsubstantiated claim. Then the documentation can be improved, either by replacing the code in the documentation which doesn't conform to best practices with code which does conform to best practices, or by removing the claim if it's just so much B.S. This helps everyone who reads the documentation in the future, and it makes for a *much* easier question to answer than "What are the best practices for developing Twisted applications?" (which I don't even know where to begin to answer, unless you've got about a week). Jean-Paul
Thanks for the answers... I will definitely file some tickets in the next few days, and the links provided have been helpful. The docs are good, it's not a quality thing at all... It's just missing that one "Here's how you put it all together" doc. It is true, I should have said "best practices". That's what i was looking for. Ah well, I'll keep chipping away... Thanks again On 6/17/08, Jean-Paul Calderone <exarkun@divmod.com> wrote:
On Tue, 17 Jun 2008 11:36:40 -0400, Black <python@blackslens.com> wrote:
On Jun 17, 2008, at 10:08 AM, Itamar Shtull-Trauring wrote:
On Tue, 2008-06-17 at 08:49 -0400, kevin beckford wrote:
Where do i read to find out how to simply create a production quality twisted application?
There are some better *deployment* techniques documented here - http://twistedmatrix.com/projects/core/documentation/howto/tap.html - but they just make things easier to deploy, they don't make your software "production quality."
Production quality software is hard; a good start is having code reviews, well-written unit tests that cover all your code, as well as end-to-end regression tests. The deployment techniques above do tend to lead to slightly more testable software (by encouraging you to write your software as Services).
Unless I misunderstood the first mail, I don't think this is what Kevin meant . I believe what he is looking for could better be termed "best practice" (it is probably the reference to "industrial grade" in the subject that is misleading). This is something I've struggled with as well
with respect to Twisted. Twisted is large and complex and in many cases there are several ways to achieve the same functionality. Frequently, it isn't easy to figure out the "best" approach or even the generally accepted approach. As Kevin says, the documentation does make obscure references to there being some "way you usually do this", without coming out and saying what that way is. All of the code reviews and unit tests in the world aren't going to catch unconventional uses of a library unless one of the reviewers knows better...
I'd offer some advice of my own, but I'm not sure even now that I use Twisted properly - at best I have something that works, and as all of my work is experimental that has been good enough.
A way to get better answers to this question is to file tickets for each place in the documentation which makes this unsubstantiated claim. Then the documentation can be improved, either by replacing the code in the documentation which doesn't conform to best practices with code which does conform to best practices, or by removing the claim if it's just so much B.S.
This helps everyone who reads the documentation in the future, and it makes for a *much* easier question to answer than "What are the best practices for developing Twisted applications?" (which I don't even know where to begin to answer, unless you've got about a week).
Jean-Paul
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Kevin Beckford Technical Lead, Lazyweb Construction Company, http://lazyweb.ca kevin@lazyweb.ca
Jean-Paul Calderone wrote:
On Tue, 17 Jun 2008 11:36:40 -0400, Black <python@blackslens.com> wrote: [...] A way to get better answers to this question is to file tickets for each place in the documentation which makes this unsubstantiated claim. Then the documentation can be improved, either by replacing the code in the documentation which doesn't conform to best practices with code which does conform to best practices, or by removing the claim if it's just so much B.S.
This helps everyone who reads the documentation in the future, and it makes for a *much* easier question to answer than "What are the best practices for developing Twisted applications?" (which I don't even know where to begin to answer, unless you've got about a week).
Nope. Sorry. Two hours and a bottle of whisky is all you get. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/
Steve Holden wrote: [...]
Nope. Sorry. Two hours and a bottle of whisky is all you get.
It's a bit late now, but this makes me think that Teach Me Twisted could have had a subtitle: “Teach Me Twisted: The blind leading the blind” -Andrew.
participants (7)
-
Andrew Bennetts -
Black -
Itamar Shtull-Trauring -
Jean-Paul Calderone -
kevin beckford -
kevin＠lazyweb.ca
-
Steve Holden