[Twisted-Python] Getting Stories Straight (RPYs)
First, let me clarify: I never said "don't use listenTCP". I merely said that using application.listenTCP(0,...) is a bad idea, and if you want to listen on an arbitrary port, you should do it with an ApplicationService. Now, let me clarify the position about RPYs: The official line is "your development must be orthogonal to your deployment". What this means is that if you are developing a web application, it should be a resource with children, and internal links. Some of the children may use Woven, some may be resources manually using .write, etc. The point is, the code should be in a Python module, or package, *outside* the web tree. You will probably want to test your application as you develop it. There are many ways to test, starting with dropping an .rpy which looks like """ from mypackage import toplevel resource = toplevel.Resource(file="foo/bar", color="blue") """ into a directory, and then running % mktap web --path=/directory % twistd -f web.tap to writing a Python script like """ #!/usr/bin/python2.2 from twisted.web import server from twisted.internet import reactor from mypackage import toplevel reactor.listenTCP(8080, server.Site(toplevel.Resource(file="foo/bar", color="blue"))) reactor.run() """ Which one of these strategies you use is not terribly important, since (and this is the important part) deployment is *orthogonal*. Later, when you want users to actually *use* your code, you should worry about what to do -- or rather, don't. Users may have widely different needs. Some may want to run your code in a different process, so they'll use distributed web. Some may be using the Debian package, and will drop in % cat > /etc/local.d/99addmypackage.py from mypackage import toplevel default.putChild("mypackage", toplevel.Resource(file="foo/bar", color="blue")) ^D If you want to be friendly to your users, you can supply many examples in your package, like the above .rpy and the Debian-package drop-in. But the *ultimate* friendliness is to write a useful resource which does not have deployment assumptions built in. This is what itamar meant by "Twisted.Web is not PHP": since we have a better tool for organizing code (Python modules and packages), use it. In PHP, the only tool for organizing is web pages, which leads to silly things like PHP pages full of functions that other pages import, etc. etc. If you translate this wholesale into Python, you may assume the correct way to do web development is to have many RPYs, all importing some Python module. This is a *bad idea* -- it mashes deployment with development, and makes sure your users will be *tied* to the file-system. We respect you enough to assume you're using Twisted.Web to write non-trivial applications. This means that you will eventually have the Twisted.Web Zen, by the time you finish to write your application. Using itamar's methodology insures that when you *do* have the Zen, you will have less of an itch to throw it away and start anew because it is crap. So, to sum up: We have .rpys because they are useful and necessary. But using them incorrectly leads to horribly unmaintainable applications. The best way to insure using them correctly is to not use them at all, until you are on your *final* deployment stages. By then, the whole "An RPY must be less than 10 lines" will be superfluous because you will not *have* more than 10 lines to write. -- Moshe Zadka -- http://moshez.org/ Buffy: I don't like you hanging out with someone that... short. Riley: Yeah, a lot of young people nowadays are experimenting with shortness. Agile Programming Language -- http://www.python.org/
On Tuesday, June 24, 2003, at 01:13 AM, Moshe Zadka wrote:
using them incorrectly leads to horribly unmaintainable applications. The best way to insure using them correctly is to not use them at all, until you are on your *final* deployment stages. By then, the whole "An RPY must be less than 10 lines" will be superfluous because you will not *have* more than 10 lines to write.
This is a great description of RPYs. In fact, I propose that we put a hack into ResourceScript which stats the .rpy file after it's been run, and if it is more than 512 bytes, print a UserWarning like: Woah there, code cowboy! It's likely you've misunderstood what RPYs are for. Have you read this? http://www.twistedmatrix.com/pipermail/twisted-python/2003-June/ 004693.html If you are quite sure that you are not committing an awful hack that will bring doom to you and shame to your family, set __no_seriously_this_rpy_needs_to_be_this_long__ = 5192 where 5192 is a new maximum size based on the current length of that file, calculated for the user's convenience to include some extra padding for the new attribute.
On Tue, Jun 24, 2003 at 01:41:05AM -0500, Glyph Lefkowitz wrote:
On Tuesday, June 24, 2003, at 01:13 AM, Moshe Zadka wrote:
using them incorrectly leads to horribly unmaintainable applications. The best way to insure using them correctly is to not use them at all, until you are on your *final* deployment stages. By then, the whole "An RPY must be less than 10 lines" will be superfluous because you will not *have* more than 10 lines to write.
This is a great description of RPYs.
In fact, I propose that we put a hack into ResourceScript which stats the .rpy file after it's been run, and if it is more than 512 bytes, print a UserWarning like:
Woah there, code cowboy! It's likely you've misunderstood what RPYs are for. Have you read this?
http://www.twistedmatrix.com/pipermail/twisted-python/2003-June/ 004693.html
[...] Except that it should refer to actual documentation, rather than the mailing list archives! I think Moshe's post is quite readily adaptable to slightly more formal documentation, though, so I volunteer to do some trivial editing and include it in the HOWTOs if people agree that's a good idea... :) -Andrew.
In fact, I propose that we put a hack into ResourceScript which stats the .rpy file after it's been run, and if it is more than 512 bytes, print a UserWarning like:
Woah there, code cowboy! It's likely you've misunderstood what RPYs are for. Have you read this?
http://www.twistedmatrix.com/pipermail/twisted-python/2003-June/ 004693.html [...]
I think we **really** need some complete doc about that. Building a large web app with half written doc is quite hard (no doc for macros, guard, Form, ...). Btw, I don't know how (and where !) to start developping my application :( RPY seems to be a good starting point to me. But I think I was using it in a wrong way. If I understand it well, I should have : - xhtml templates. - xhtml macro templates eventually. - rpy files. - py files. xhtml templates are used for presentation python files are used for logic rpy are glue between presentation and logic. that's it ?
On 24 Jun 2003, Philippe <lafou@wanadoo.fr> wrote:
RPY seems to be a good starting point to me.
It isn't. That was itamar's point: don't use RPYs. It's not that they are not useful, but... OK, here's an analogy. When teaching kids correct table manners, you put books under their arms, and tell them not to let the books fall. It's not that being able to hold books while eating is a useful skill -- it's that the books force the kid to constrain the movements of his hands. In a similar way, not using RPYs will force you to think about the correct way to build your application. If you feel yourself forced to use RPY *avoid the temptation*, and know that what you're feeling is actually a way to build the application *incorrectly*. If you're still not sure what to do, ask advice -- here or on #twisted. But in no case succumb to the "just write RPYs". You should only use RPY when you are able to build applications without RPYs. This is why you see tools like Issues, say, use RPYs -- notice how you *can* use Issues *without* RPYs.
If I understand it well, I should have :
- xhtml templates. - xhtml macro templates eventually. - rpy files. - py files.
NO! You should *not* have RPY files Some Python modules will be used for pure, non-web-related business logic. Some Python modules will define resource trees which bind together logic and presentation, for example by using Page instances and defining wchild_* methods. Perhaps you'll find a need to write an ApplicationService which holds various resources, usually not. Notice the one thing missing from this paragraph: .rpy files. That's right, you should not have *any* .rpy files. You *may* have need for Python modules with long and tortorous functions like: def buildBlogResource( realName, numberOfArticles, numberOfLines, colorScheme, .... ): blogObject = blog.DataRespository(...) root = blog.rootResource(blogObject) showArticles = blog.Show(blogObject) root.putChild('show', showArticles) # Notice how I use relative URLs here: updateArticlesForm = blog.UpdateForm(blogObject, url="doupdate") root.putChild('update', updateArticles) doUpdate = blog.DoUpdate(blogObject) root.putChild('doupdate', doUpdate) return root But there should be *no* RPYs. Now, when testing, you might use a script similar to """ #!/usr/bin/python2.2 from blog import buildBlogResource from twisted.web import server from twisted.internet import reactor resource = buildBlogResource("Moshe", 10, 20, "RED", ...) reactor.listenTCP(8080, server.Site(resource)) """ When you are done testing, you package your Python library. Now, you may do several things: 1. Put a .rpy in your doc/examples which looks something like """ # change these lines to configure name = "Moshe" articles = 10 lines = 20 colorScheme = "RED" ... # touch nothing below this line from blog import buildBlogResource resource = buildBlogResource(name, articles, lines, colorScheme, ...) """ 2. Write a TAP plugin to easily generate blogs. See plugin.html for full details, I'll show an excerpt here: """" from blog import buildBlogResource def updateApplication(app, config): name, articles, lines, colorScheme = (config['name'], config['articles'], config['lines'], config['colorScheme]') resource = buildBlogResource(name, articles, lines, colorScheme, ...) app.listenTCP(config['port'], server.Site(resource)) """ Your users can now do % mktap myblog --name=Moshe --colorScheme=RED % twistd -f myblog 3. Supply a twisted-web Debian plugin, which looks like """ # change these lines to configure name = "Moshe" articles = 10 lines = 20 colorScheme = "RED" ... # touch nothing below this line from blog import buildBlogResource resource = buildBlogResource(name, articles, lines, colorScheme, ...) default.putChild('myblog', resource) """ Sitting in /etc/twisted-web/local.d/99myblog.py 4. Document buildBlogResource in your API docs, so savvy Twisted programmers can deliver more and better ways to integrate it with a Python app. For example, writing through-the-web management to add a blog to a site. 5. Write TRPs (Twisted Resource Pickles) command-line generators. etc. etc. etc. Please notice when the RPY arrived on the scene: a long time after you had a fully functioning blog, and you were *documenting* it. This is the proper time to think about deployment issues. -- Moshe Zadka -- http://moshez.org/ Buffy: I don't like you hanging out with someone that... short. Riley: Yeah, a lot of young people nowadays are experimenting with shortness. Agile Programming Language -- http://www.python.org/
On Tue, Jun 24, 2003 at 09:44:47AM -0000, Moshe Zadka wrote:
On 24 Jun 2003, Philippe <lafou@wanadoo.fr> wrote:
RPY seems to be a good starting point to me.
It isn't. That was itamar's point: don't use RPYs. It's not that they are not useful, but...
[...]
When you are done testing, you package your Python library. Now, you may do several things:
1. Put a .rpy in your doc/examples which looks something like [...]
Please notice when the RPY arrived on the scene: a long time after you had a fully functioning blog, and you were *documenting* it. This is the proper time to think about deployment issues.
Just to elaborate a little on what Moshe said, notice what function the RPY in Moshe's example is serving here: *it's a configuration file*. It's not a place for arbitrary code and logic, it's merely somewhere you configure a predefined Resource class. [Someone please correct me if I'm wrong!] -Andrew.
On Tue, 24 Jun 2003, Andrew Bennetts <andrew-twisted@puzzling.org> wrote:
Just to elaborate a little on what Moshe said, notice what function the RPY in Moshe's example is serving here: *it's a configuration file*. It's not a place for arbitrary code and logic, it's merely somewhere you configure a predefined Resource class.
[Someone please correct me if I'm wrong!]
Well, in my example, it was a configuration file. It could easily have been the place of "arbitrary" code: """ d = {} execfile("/etc/myblog.py", d) from blog import buildBlogResource resource = buildBlogResource(**d) """ And have /etc/myblog.py look like """ name="Moshe" colorScheme="RED" articles=10 lines=20 """ The .rpy file, more than a configuration file, is a "place indicator" -- it is a link back from the filesystem (which, in the Twisted.Web view, is just a particularily ugly resource hierearchy) back into a "nice" resource hierarchy. It's just that the filesystem is very easy to use for "static" resources. The RPY file is a *deployment* choice. Deployment often ties in with configuration: sometimes intimately (as in my original example) and sometimes not so intimately (like in this example, where the configuration is in a different file). You can also do even less trivial things in the RPY: """ import ConfigParser config = ConfigParser.ConfigParser("/etc/myblog.ini") from blog import buildBlogResource resource = buildBlogResource(config.get("name"), config.getint("articles"), config.getint("lines"), config.get("colorScheme")) """ You can choose whether to put cache() at the top or not, etc. etc. -- Moshe Zadka -- http://moshez.org/ Buffy: I don't like you hanging out with someone that... short. Riley: Yeah, a lot of young people nowadays are experimenting with shortness. Agile Programming Language -- http://www.python.org/
On Tue, Jun 24, 2003 at 04:55:48PM +1000, Andrew Bennetts wrote:
On Tue, Jun 24, 2003 at 01:41:05AM -0500, Glyph Lefkowitz wrote:
http://www.twistedmatrix.com/pipermail/twisted-python/2003-June/ 004693.html
[...]
Except that it should refer to actual documentation, rather than the mailing list archives! I think Moshe's post is quite readily adaptable to slightly more formal documentation, though, so I volunteer to do some trivial editing and include it in the HOWTOs if people agree that's a good idea... :)
I've now done this -- you can see the result, fresh off the buildbot, at: http://twistedmatrix.com/users/warner/doc-latest/howto/web-development.xhtml As with all of our docs, feedback is welcome! -Andrew.
On Tue, Jun 24, 2003 at 11:29:12PM +1000, Andrew Bennetts wrote:
I've now done this -- you can see the result, fresh off the buildbot, at: http://twistedmatrix.com/users/warner/doc-latest/howto/web-development.xhtml
As with all of our docs, feedback is welcome!
One of the nice things about .rpy is that they work well with the usual HTML development tweak-source-reload-browser cycle. The (excellent and informative) document above says "The best way to ensure you are using [.rpys] correctly is to not use them at all until you are on your final deployment stages" - I feel it might be nice to mention, if only briefly or in a footnote or at the other end of a hyperlink, the Twisted Way of debugging a live web app. -- ___________ ____________________________ | Screwtape | Reply-To: munged on Usenet |________ ______ ____ __ _ _ _ | | "The doctor walked in, sminking of gin... he was really sminking." -- McCartney |
On Tue, 2003-06-24 at 11:54, screwtape@froup.com wrote:
One of the nice things about .rpy is that they work well with the usual HTML development tweak-source-reload-browser cycle.
This is true - most web developers are not used to having to stop and restart the server after every change. Is there anything wrong with putting all your code into /index.rpy during development, so that code gets reloaded on every page visit? It's easy enough to move the code into a regular module later. Abe
On 24 Jun 2003, Abe Fettig <abe@fettig.net> wrote:
This is true - most web developers are not used to having to stop and restart the server after every change. Is there anything wrong with putting all your code into /index.rpy during development, so that code gets reloaded on every page visit? It's easy enough to move the code into a regular module later.
Depends on how much code that is. Of course, you can also do that if you don't put all your code in the .rpy, by having a short .rpy which reloads modules: ''' import blog reload(blog) resource = blog.buildBlogResource(...) ''' is a perfectly sane short RPY which will allow you to code properly. However, I tend to dislike this kind of solutions. While in CGI, this kind of strategy makes sense, in real web applications, it starts to break down. My vague memories of using HTML::Mason and OpenACS indicate that at one point I was so tired of remembering which parts were cached and which parts weren't that I actually did restart the web server whenever I wanted to test my changes. I usually do my web development with a web browser open, and whenever I change I restart the server, and reload the page. Moving from two clicks per change to one doesn't seem worth remembering what's cached and what isn't, sorry... :) -- Moshe Zadka -- http://moshez.org/ Buffy: I don't like you hanging out with someone that... short. Riley: Yeah, a lot of young people nowadays are experimenting with shortness. Agile Programming Language -- http://www.python.org/
On 2003.06.24 14:15, Abe Fettig wrote:
On Tue, 2003-06-24 at 11:54, screwtape@froup.com wrote:
One of the nice things about .rpy is that they work well with the usual HTML development tweak-source-reload-browser cycle.
This is true - most web developers are not used to having to stop and restart the server after every change. Is there anything wrong with putting all your code into /index.rpy during development, so that code gets reloaded on every page visit? It's easy enough to move the code into a regular module later.
I would strongly recommend, rather than shoving all of your code into an .rpy file for its reloading semantics, just calling reload() (or twisted.python.rebuild.rebuild()) on the modules that are part of your project in the .rpy file. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://twistedmatrix.com/users/radix.twistd/
Greetings, Sorry about the newbie question, but... :) I have a custom server running a byte-oriented protocol (sockets, written in C) and would like to use twisted to implement the client. I see from looking at the "simpleclient" how you can use an existing protocol (e.g. Echo) but I am not sure how to create a client using my own byte-oriented (e.g. "raw") protocol. If anyone could give me a little advice I would certainly appreciate it. :) Also, is ClientFactory appropriate for this sort of application? Cheers, Don
On Tue, Jun 24, 2003 at 06:13:29PM -0500, Don Hiatt wrote:
Greetings,
Sorry about the newbie question, but... :)
I have a custom server running a byte-oriented protocol (sockets, written in C) and would like to use twisted to implement the client. I see from looking at the "simpleclient" how you can use an existing protocol (e.g. Echo) but I am not sure how to create a client using my own byte-oriented (e.g. "raw") protocol. If anyone could give me a little advice I would certainly appreciate it. :) Also, is ClientFactory appropriate for this sort of application?
Well, the 'data' parameter of an Protocol's dataReceived method is just a string, i.e. bytes directly from the network. I've used Twisted in the past for working with byte stream protocols -- it works very nicely. A key part is the 'struct' module in Python's standard library, which can convert bytes to numbers, and so forth. Take a look at the source to twisted.protocols.basic.Int32Receiver, which receives strings prefixed by a 32-bit length. There's really nothing to it :) Regarding ClientFactory -- yes, whenever you are using clients, you probably want some sort of ClientFactory (quite possibly a ReconnectingClientFactory, depending on the application). See http://twistedmatrix.com/documents/howto/clients for an overview of using ClientFactory. -Andrew.
This is exactly what I was looking for, thanks Andrew! Cheers, don P.S. Thanks also for your previous hint on restricting the web server to single connections. -----Original Message----- From: twisted-python-admin@twistedmatrix.com [mailto:twisted-python-admin@twistedmatrix.com]On Behalf Of Andrew Bennetts Sent: Tuesday, June 24, 2003 9:00 PM To: twisted-python@twistedmatrix.com Subject: Re: [Twisted-Python] Help creating a client for a custom protocol On Tue, Jun 24, 2003 at 06:13:29PM -0500, Don Hiatt wrote:
I have a custom server running a byte-oriented protocol (sockets, written in C) and would like to use twisted to implement the client. I see from looking at the "simpleclient" how
A key part is the 'struct' module in Python's standard library, which can convert bytes to numbers, and so forth. Take a look at the source to twisted.protocols.basic.Int32Receiver, which receives strings prefixed by a 32-bit length. There's really nothing to it :)
On Tue, Jun 24, 2003 at 06:38:59PM -0400, Christopher Armstrong wrote:
On 2003.06.24 14:15, Abe Fettig wrote:
On Tue, 2003-06-24 at 11:54, screwtape@froup.com wrote:
One of the nice things about .rpy is that they work well with the usual HTML development tweak-source-reload-browser cycle.
This is true - most web developers are not used to having to stop and restart the server after every change. Is there anything wrong with putting all your code into /index.rpy during development, so that code gets reloaded on every page visit? It's easy enough to move the code into a regular module later.
I would strongly recommend, rather than shoving all of your code into an .rpy file for its reloading semantics, just calling reload() (or twisted.python.rebuild.rebuild()) on the modules that are part of your project in the .rpy file.
Note that there are now docs on rebuild in CVS, too: http://twistedmatrix.com/users/warner/doc-latest/howto/upgrading.xhtml -Andrew.
I've now done this -- you can see the result, fresh off the buildbot, at: http://twistedmatrix.com/users/warner/doc-latest/howto/web-development.xhtml
hmm make things a little more clearer. But it still looks like an intro for me : How can I develop a large woven application ?? I don't see how to make links between pages (resources). I know I can define some new Child resource, but how do manage several Pages ? Will I have to do a from MyResources import myresource1, myresource2 ??
On 2003.06.24 12:14, Philippe Lafoucrière wrote:
I don't see how to make links between pages (resources). I know I can define some new Child resource, but how do manage several Pages ?
You should have a single root resource that knows the layout of your entire web app. Once you've got that, the only kinds of links you need are child and sibling. Also, if you set appRoot = True in your root Page subclass, you can call request.getRootURL() anywhere else to get a URL that points to the root of your web app. You would manage several Pages by making them all children of your main app-root Page. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://twistedmatrix.com/users/radix.twistd/
participants (8)
-
Abe Fettig -
Andrew Bennetts -
Christopher Armstrong -
Don Hiatt -
Glyph Lefkowitz -
Moshe Zadka -
Philippe Lafoucrière -
screwtape＠froup.com