Twisted & legacy code - Unorthodox? Opinions wanted
Apologies for a long post. Felt it was needed to get the message across. What's described could be a creative approach to dealing with legacy code, but it could also be an ugly hack, and I would really benefit from the views of you gurus as to whether this looks like a good/acceptable/bad way to use Twisted. (If 'bad' - any immediate suggestions? :)) Thanks in advance, Harald We have an existing web app (~10 000 lines of Python, generating HTML, with 25 cgi scripts), running on Apache. We are considering Twisted as one alternative to make it faster and better structured. Just running the existing code with Twisted instead of Apache doesn't increase performance much, as CGI's process startups are the bottleneck. So, to really gain from Twisted, I guess we need to make the code event-driven (and this would be the ultimate goal). Now, one design has been proposed, which boosts performance dramatically with few code changes. It uses Twisted as a plain TCP socket server, communicating with text strings instead of using HTTP. The browser-server interaction is - - Request: "scriptSelectorKeyword::formInputDict::cookiesDict" - Response: "generatedHTML" (as before, but with no line breaks) Creating the request in the browser: All links in the page are modified to NOT trigger a GET/POST, but use javascript to - scan form input (<INPUT>, <SELECT>, etc) and build formInputDict ("{name: value, ...}") - create a similar representation of the cookies, as cookiesDict - send the "::"-delimited string using a small applet The server (having imported all modules used by the app at startup) - - evals formInputDict and cookiesDict (since they come as text strings) - based on scriptSelectorKeyword, invokes the appropriate module in a thread (threads.deferToThread) - (we don't use CGI, so formInputDict and cookiesDict replace Python's cgi & Cookies modules) - the module has a "getBuffer" method that returns the generated HTML instead of printing it - the generated HTML is written to the socket (reactor.callFromThread(self.transport.write, html)) The browser - receives the generated HTML string through the applet - parses the cookies - displays the HTML, using document.write This design is up and running with impressive performance, and the users are enthusiastic. However, the maintenance team is less so, and these are our main concerns (with the advocates' answers added) - Is this really a valid way to use Twisted? - Think so. The docs say, "Twisted still allows the use of threads if you really need them, usually to interface with synchronous legacy code", but we should ask the experts. Will it scale? Will it work with future Twisted versions? - Hope so. See previous point. It's not a web app anymore, it's a javascript app. - So? It locks us into MS Internet Explorer. - Yes. However, MSIE is our corporate standard anyway. It can't be used through a firewall. - No. Luckily, this is a strictly internal app. We don't want to rely on and maintain home-written javascript. - Good point. However, 1) We're talking about roughly 2 A4 pages 2) The javascript to handle form input & cookies is meant to be general, and once it's validated, we should not need to touch it much. 3) If we accept being tied in with MSIE, ain't its javascript part of the package? We can't do unit tests on the javascript. - Correct. See previous point.
On Thu, 2004-04-22 at 08:22, Harald.Nyman@astrazeneca.com wrote:
What's described could be a creative approach to dealing with legacy code, but it could also be an ugly hack, and I would really benefit from the views of you gurus as to whether this looks like a good/acceptable/bad way to use Twisted. (If 'bad' - any immediate suggestions? :))
Certainly.
Creating the request in the browser: All links in the page are modified to NOT trigger a GET/POST, but use javascript to - scan form input (<INPUT>, <SELECT>, etc) and build formInputDict ("{name: value, ...}") - create a similar representation of the cookies, as cookiesDict - send the "::"-delimited string using a small applet
Unless you are being _very_ selective and _very_ careful in how this dictionary is formed and interpreted, it seems like this might open up unwarranted vulnerabilities.
The browser - receives the generated HTML string through the applet - parses the cookies - displays the HTML, using document.write
Yikes!
Is this really a valid way to use Twisted?
I would say any use of Twisted is a valid use, but that doesn't mean it's the best way to leverage the library's real strengths. "Luke, if only you knew the power..."
Will it scale? Will it work with future Twisted versions?
Probably, but again, you're not really taking advantage of all the flexibility and modularity that Twisted offers. You're force-feeding the server in order to squeeze out some performance benefits, and doing so in a way that (to me) seems destined to become a developer's nightmare.
It can't be used through a firewall.
See below for an application design that _can_ go through a firewall.
We don't want to rely on and maintain home-written javascript. - Good point. However, 1) We're talking about roughly 2 A4 pages
It will grow.
2) The javascript to handle form input & cookies is meant to be general, and once it's validated, we should not need to touch it much.
Unless security issues come along. And they will.
We can't do unit tests on the javascript. - Correct. See previous point.
False. Unit tests are entirely possible in Javascript. Whether you actually _want_ to do them is a different issue. Hint: you don't. :) You should take a look at how Nevow <http://www.nevow.com/> has implemented its Freeform module. I'm not saying you need to port your entire app to Nevow (though it is worth considering), but I think you're trying to do roughly the same thing. Maybe by emulating some of Nevow's design decisions, you could get equal performance benefits with a lot less Javascript magic. An example query: 1. You have a Twisted server running, which serves up web pages. For example, going to /tasks will show you your task list for today. (I'm pretending I know anything about what your web app does.) 2. The HTML code for /tasks renders a form, "foo", with a checklist or a bunch of text fields or whatever else you want. 3. You type in some fields, change values, uncheck stuff, whatever. Click the Submit button, and it sends a POST to /tasks/formpost!foo. 4. Your Twisted server, receiving this POST, attempts to find the Python object responsible for handling the URL /tasks/formpost!foo: a. First it finds "/", your site root, and asks for "tasks". b. Next finds "tasks", which is your TasksPage instance (or something like that). Twisted asks the TasksPage instance to find a child named "formpost!foo". c. Your TasksPage says, "AHA! I recognize this! It's not really requesting a separate page. That weird URL tells me to interpret the form data, modify some internal state or business objects, and then go right back to the tasks page where I started. Done!" 5. Profit. This is, with a _lot_ of technical detail missing, what Freeform is all about -- designing a process for designing objects for which forms can be automatically rendered and interpreted, with minimal effort on your part. I'm already using it to design a non-fancy CMS that will power my web page very soon. Nevow is currently undergoing some serious reorganization, and after the refactor, Freeform will have gone from a really cool gizmo to a very strong, flexible engine. If you're interested in using such a framework, I suggest getting more information from those developers more closely involved in the process. (If you don't have an aversion to IRC, the best place to get the latest scoop is in #twisted.web on irc.freenode.net.) However, if your team isn't comfortable working with a platform that is still going through serious changes, just implementing the above process on your own would, in my view, result in a much better platform for web development than the Javascript/socket mess you've suggested above. One man's opinion, writ large. Anyone else? -- Alex Levy WWW: http://mesozoic.geecs.org/ "Never let your sense of morals prevent you from doing what is right." -- Salvor Hardin, Isaac Asimov's _Foundation_
participants (3)
-
Alex Levy -
Harald.Nyman＠astrazeneca.com -
Stephen Thorne