Turkey Questions 1-5
Hello, I've changed the thread from "Re: Introduction, newbie confusion plus an offer" to "Turkey Questions 1-5" to start the Q&A. Goal: Create the world's simplest web-server My first set of questions are based on "Configuring and Using the Twisted.Web Server in "Twisted.Web Documentation." Inspired by the section "Installing a pre-configured server," I did the following: -- created the directory ~/twisted/www -- created the file ~/twisted/www/myHTML.html <html> <head> <title>flat html file</title> </head> <body> <h1>Hello, World!</h1> </body> </html> -- entered "mktap web --path ~/twisted/www --logfile ~/twisted/www --port 8080" into the command line -- entered "twistd -f web.tap" into command line Results: -- System wrote the file "web.tap" into ~$, my home directory -- System wrote twistd.log into ~/twisted/www -- When I entered "http://localhost:8080/myHTML.html, my browser displayed "Hello, World!" OK, success. Questions: 1) Why did mktap write web.tap to my home directory rather than ~/twisted/www? 2) Why didn't myHTML.html require a "Content-type: text/html" line? 3) How can I see the source for the server created by mktap? 4) Are there preferred practices for creating directories for the document root (for instance, better in user or root), directories for resource modules, site data, css styling, etc.? 5) Are there preferred practices for assigning permissions to files in the above directories? Best wishes, Lloyd
lloyd@paisite.com wrote:
Hello,
I've changed the thread from "Re: Introduction, newbie confusion plus an offer" to "Turkey Questions 1-5" to start the Q&A.
Goal: Create the world's simplest web-server
My first set of questions are based on "Configuring and Using the Twisted.Web Server in "Twisted.Web Documentation."
Inspired by the section "Installing a pre-configured server," I did the following:
-- created the directory ~/twisted/www -- created the file ~/twisted/www/myHTML.html
<html> <head> <title>flat html file</title> </head> <body> <h1>Hello, World!</h1> </body> </html>
-- entered "mktap web --path ~/twisted/www --logfile ~/twisted/www --port 8080" into the command line -- entered "twistd -f web.tap" into command line
Results:
-- System wrote the file "web.tap" into ~$, my home directory -- System wrote twistd.log into ~/twisted/www -- When I entered "http://localhost:8080/myHTML.html, my browser displayed "Hello, World!"
OK, success.
Questions:
1) Why did mktap write web.tap to my home directory rather than ~/twisted/www?
Presumably because you ran mktap from your home directory. The --path is the path to the root of the web server. The .tak file is written to the current directory.
2) Why didn't myHTML.html require a "Content-type: text/html" line?
The web.tap you created actually uses a twisted.web.static.File instance to serve the files from the --path directory. static.File guesses the MIME type and sends the Content-Type response header to the browser.
3) How can I see the source for the server created by mktap?
Good question and, to be honest, I rarely use mktap. I find it useful for quickly creating a web server for static content but most of the stuff I use is highly dynamic. Here's an equivalent .tac file that serves static content: from twisted.application import internet, service from twisted.web import server, static WWWROOT = '/home/matt/www/docs' LOGPATH = '/home/matt/www/logs/access.log' PORT = 8080 application = service.Application('web-server') site = server.Site(static.File(WWWROOT), logPath=LOGPATH) internet.TCPServer(PORT, site).setServiceParent(application) That sort of thing can be easily modified into something that creates a dynamic root resource.
4) Are there preferred practices for creating directories for the document root (for instance, better in user or root), directories for resource modules, site data, css styling, etc.?
I never use .rpy resources so all my Python is in packages and modules. I tend to have templates, css, js and images in a directory somewhere and a .tac to start the services. Site data is either in a database or in another directory, away from the templates etc.
5) Are there preferred practices for assigning permissions to files in the above directories?
Well, I think the most important thing is to avoid making *any* code or data directly accessible from the web. That's fairly easy to do if you don't use .rpy files and make sure that any directories served by static.File resources don't contain any files with sensitive content. XHTML templates should not be directly accessible over the web either. Hope some of that is useful. - Matt -- __ / \__ Matt Goodall, Pollenation Internet Ltd \__/ \ w: http://www.pollenation.net __/ \__/ e: matt@pollenation.net / \__/ \ t: +44 (0)113 2252500 \__/ \__/ / \ Any views expressed are my own and do not necessarily \__/ reflect the views of my employer.
Hi Matt, Thanks for responding. I've started a website to capture the sequence of experiments, questions, and responses. Will have posted this weekend.
Presumably because you ran mktap from your home directory. The --path is the path to the root of the web server. The .tak file is written to the current directory.
Makes sense.
2) Why didn't myHTML.html require a "Content-type: text/html" line?
The web.tap you created actually uses a twisted.web.static.File instance to serve the files from the --path directory. static.File guesses the MIME type and sends the Content-Type response header to the browser.
ok. I rarely use mktap. I find it useful
for quickly creating a web server for static content but most of the stuff I use is highly dynamic.
Here's an equivalent .tac file that serves static content:
from twisted.application import internet, service from twisted.web import server, static
WWWROOT = '/home/matt/www/docs' LOGPATH = '/home/matt/www/logs/access.log' PORT = 8080
application = service.Application('web-server') site = server.Site(static.File(WWWROOT), logPath=LOGPATH) internet.TCPServer(PORT, site).setServiceParent(application)
I copied and saved this code to web.tac. Then tried to launch it with twistd -ny web.tac. System came back with response: Failed to load application: TCPServer has no attribute 'setServerParent' I looked at source for twisted.application.internet, which seems to import new to synthesize TCPServer. Couldn't follow from there because I didn't have time to track down new. Question 6: Why did this code fail? Question 7: Where is new?
That sort of thing can be easily modified into something that creates a dynamic root resource.
This is the line I'd like to pursue next. I'll see how far I can go on my own, then come back when stuck.
I never use .rpy resources so all my Python is in packages and modules. I tend to have templates, css, js and images in a directory somewhere and a .tac to start the services.
Question 8: What's wrong with .rpy resources?
Site data is either in a database or in another directory, away from the templates etc.
ok
Well, I think the most important thing is to avoid making *any* code or data directly accessible from the web. That's fairly easy to do if you don't use .rpy files and make sure that any directories served by static.File resources don't contain any files with sensitive content. XHTML templates should not be directly accessible over the web either.
Got it. Again, thanks for you help, Matt. Best wishes, Lloyd
On Fri, 8 Jul 2005 16:33:51 -0400 (EDT), lloyd@paisite.com wrote:
Hi Matt,
Thanks for responding.
I've started a website to capture the sequence of experiments, questions, and responses. Will have posted this weekend.
[snip] Question 7: Where is new?
It's a standard library module. foo = new.classobj('foo', (Bar, Baz), {'quux': foobar}) Is the same as: class foo(Bar, Baz): quux = foobar The code in question is in bad need of a refactoring. Jp
Hi all, As promised last week, I've posted a site for supplementary documentation of Twisted -- e.g. documentation for people like me who really need their hands held as they try to understand and use Twisted. The URL: http://twisted.paisite.com The goal is twofold: 1) To accelerate my own learning curve through Twisted and related tools 2) To give something back to the incredible community that has conceived and implemented Twisted and related frameworks. Since I don't have much to offer technically, the least I can do is to take on the role of tech-challenged user, ask the many naive questions on issues that experienced users may take for granted, and try to fill in the gaps in documentation. The approach will be a series of experiments, taken in small steps, each designed to build knowledge and confidence. This approach is inspired largely by the excellent "Twisted from Scratch, or the Evolution of Finger" tutorials. I'll do my best to dig out what I can from existing documentation and other sources, but I can only succeed with your help. I hope my many stupid questions don't come to annoy you; if so, please let me know. Meanwhile, I welcome your input and criticism at any and all levels. All the best, Lloyd R. Prentice P.S. It may take a day or two for the response e-mail address in the site to come live.
Thanks very much, Tom and Matt, Tom Carmichael wrote:
I think you have a typo.
Took awhile for my aging eyes to see that I'd typed "setServerParent" rather than "setServiceParent." And thanks Matt Goodall for the code fragment. Got it working. Now need to figure out how to build on it. It'll be the basis for the next Turkey experiment. Best, Lloyd
Hello everybody. I have been using the livepage-completion-notification-X branches of nevow and came across an issue with Internet Explorer. In the liveglue.js there is a place where it tries to add some data to the xmlhttp request object. IE doesn't allow this. My solution was to make the createRequest() function return a new js object rather than the xmlhttprequest itself. It seems to be working, but I am not a js guru by any stretch of the imagination so I have no idea if this is a good way of going about solving the issue. I'll paste my version here for review. ########## var liveevil_unload = false; var auto_open = true; var base_url = this.location.toString() var queryParamIndex = base_url.indexOf('?') if (queryParamIndex != -1) { base_url = base_url.substring(0, queryParamIndex) } if (base_url.charAt(base_url.length-1) != '/') base_url += '/' function createRequest() { if (window.XMLHttpRequest) { req = new XMLHttpRequest() } else { req = new ActiveXObject("Microsoft.XMLHTTP") } reqObj = new Object() reqObj.request = req return reqObj } var last_request = null var last_server_message_time = null function connect(outputNum) { var xmlhttp = createRequest() last_request = xmlhttp.request xmlhttp.request.onreadystatechange = function() { if (xmlhttp.request.readyState == 4) { if (xmlhttp.request.responseText) { last_server_message_time = new Date() eval(xmlhttp.request.responseText) if (!liveevil_unload && auto_open) { connect(outputNum + 1) } } else { last_request = null } } } xmlhttp.request.open("GET", base_url + "nevow_liveOutput?outputNum=" + outputNum + "&client-handle-id=" + nevow_clientHandleId, true) xmlhttp.request.send(null) } var userAgent = navigator.userAgent.toLowerCase() if (userAgent.indexOf("msie") != -1) { /* IE specific stuff */ /* Abort last request so we don't 'leak' connections */ window.attachEvent("onbeforeunload", function() { if (last_request ! = null) {last_request.abort();} } ) /* Set unload flag */ window.attachEvent("onbeforeunload", function() { liveevil_unload = true; } ) } else if (document.implementation && document.implementation.createDocument) { /* Mozilla specific stuff (onbeforeunload is in v1.7+ only) */ window.addEventListener("beforeunload", function() { liveevil_unload = true; }, false) } if (auto_open) { connect(0) } var inputListeners = [] var listenerId = 0 function listener(callWhenAllDone) { this.listenerId = listenerId listenerId += 1 this.events = [] this.callWhenAllDone = callWhenAllDone this.fired = false this.inputDone = function(what) { var found = false for (var i in this.events) { if (this.events[i] == what) { this.events.splice(i, 1) found = true break } } if (this.events.length == 0) { if (this.fired) { alert("Tried to fire twice :(") } else { this.callWhenAllDone() this.fired = true } } } } function listenForInputEvents(callWhenAllDone) { var newListener = new listener(callWhenAllDone) inputListeners.push(newListener) return newListener } function stopListening(theListener) { for (var i in inputListeners) { if (inputListeners[i] == theListener) { inputListeners.splice(i, 1) break } } if (theListener.events.length == 0) { theListener.callWhenAllDone() } } function nevow_clientToServerEvent(theTarget, evalAfterDone) { if (theTarget != 'close' && liveevil_unload) { // Server had previously closed the output; let's open it again. if (auto_open) { liveevil_unload = false } connect(0) } var additionalArguments = '' for (i = 2; i<arguments.length; i++) { additionalArguments += '&arguments=' additionalArguments += encodeURIComponent(arguments[i]) } var input = createRequest() input.request.onreadystatechange = function() { if (input.request.readyState == 4) { eval(input.request.responseText) if (evalAfterDone) { eval(evalAfterDone) } for (var i in input.events) { input.events[i].inputDone(input.request) } } } input.events = [] for (var i in inputListeners) { inputListeners[i].events.push(input.request) input.events.push(inputListeners[i]) } input.request.open( "GET", base_url + "nevow_liveInput?handler-path=&handler-name=" + encodeURIComponent(theTarget) + '&client-handle-id=' + nevow_clientHandleId + additionalArguments) input.request.send(null) } function nevow_setNode(node, to) { document.getElementById(node).innerHTML = to; } function nevow_appendNode(node, what) { var oldnode = document.getElementById(node); var newspan = document.createElement('span'); newspan.innerHTML = what; for (i=0; i<newspan.childNodes.length; i++) { oldnode.appendChild(newspan.childNodes[i]); } } function nevow_prependNode(node, what) { var oldnode = document.getElementById(node); var newspan = document.createElement('span'); newspan.innerHTML = what; for (i=newspan.childNodes.length-1; i>=0; i--){ if (oldnode.childNodes.length == 0) oldnode.appendChild(newspan.childNodes[i]); else oldnode.insertBefore(newspan.childNodes[i], oldnode.childNodes[0]); } } function nevow_insertNode(node, before) { var oldnode = document.getElementById('before'); var newspan = document.createElement('span'); newspan.innerHTML = what; var previous = oldnode; for (i=0; i<newspan.childNodes.length; i++) { previous.parentNode.insertBefore(newspan.childNodes[i], previous); previous = newspan.childNodes[i]; } } function nevow_closeLive(evalAfterDone) { // Tell connect() not to complain at us when the server closes the // connection with no serverToClientEvent liveevil_unload = true var old_auto_open = auto_open auto_open = false // Tell the server we know we're done, send us an empty event // evalAfterDone will be evalled after the server sends us an empty event nevow_clientToServerEvent('close', '', evalAfterDone) auto_open = old_auto_open } var server = { handle: function(handlerName) { var args = [handlerName, ''] for (var i = 1; i < arguments.length; i++) { args.push(arguments[i]) } nevow_clientToServerEvent.apply(this, args) } }
Hello, Question 9: I've been using the Debian stable packages Twisted 1.3.0-8. Is there a more current version? Question 10: If so, does it make sense to upgrade? Question 11: Is there a Debian package for the upgrade? Question 12: If not, how do I upgrade (specific steps, please) Thanks all, Lloyd
On Tue, 12 Jul 2005 14:53:29 -0400 (EDT), lloyd@paisite.com wrote:
Hello,
Question 9:
I've been using the Debian stable packages Twisted 1.3.0-8. Is there a more current version?
Yes. 2.0.1 is out. There are debian packages available here - <http://twistedmatrix.com/~tv/2.0-BEWARE>. While I won't suggest that you shouldn't "beware" of the quality of these packages, I will point out that they've been out for some time now and I haven't heard of any problems with them.
Question 10:
If so, does it make sense to upgrade?
If you're just getting started (as the case seems to be for you), yes, definitely. If you had some old software based on 1.3, there might be something to discuss, but since you're starting fresh I see no point in not starting with the latest release.
Question 11:
Is there a Debian package for the upgrade?
Question 12:
If not, how do I upgrade (specific steps, please)
I'll leave these to someone with more details (Tommi?), but I didn't have any difficulty installing the packages. Jp
lloyd@paisite.com wrote:
Question 9:
I've been using the Debian stable packages Twisted 1.3.0-8. Is there a more current version?
You'll find 2.0 in unstable/testing. Of course, using anything that explicitly claims to changing a lot is a step not to be taken lightly.
Question 10:
If so, does it make sense to upgrade?
There should be no pressing need, unless you hit some of the bugs which have already been fixed. Writing new applications or documents based on the old version is a somewhat bad decision. I suggest you keep using debian stable for now, install subversion, check out the development tree with svn co svn://svn.twistedmatrix.com/svn/Twisted/trunk and set the environment variable PYTHONPATH to point to that when working on twisted. That way, you have a stable basic version, but still can see e.g. fixes and additions as soon as they are available.
Question 11:
Is there a Debian package for the upgrade?
In unstable and testing. I currently have very little interest in providing explicit backports to stable, but the package will probably install just fine in stable -- atleast for now.
Wed, Jul 13, 2005 at 02:06:13PM +0300, Tommi Virtanen пишет:
Question 11:
Is there a Debian package for the upgrade?
In unstable and testing. I currently have very little interest in providing explicit backports to stable, but the package will probably install just fine in stable -- atleast for now.
No, at least apt-proxy will get broken - the apt-proxy package depends on twisted 1.3. -- Eugene
Hi Tommi,
You'll find 2.0 in unstable/testing. Of course, using anything that explicitly claims to changing a lot is a step not to be taken lightly.
Thanks. Will have to wait two years for the testing version to be moved over to stable?
There should be no pressing need, unless you hit some of the bugs which have already been fixed.
Writing new applications or documents based on the old version is a somewhat bad decision.
I suggest you keep using debian stable for now, install subversion, check out the development tree with svn co svn://svn.twistedmatrix.com/svn/Twisted/trunk and set the environment variable PYTHONPATH to point to that when working on twisted. That way, you have a stable basic version, but still can see e.g. fixes and additions as soon as they are available.
Sounds like a good idea. I'm not up on subversion, but it's on my list of skills to master. Thanks for the help. Lloyd
On Thu, 7 Jul 2005 16:03:31 -0400 (EDT), lloyd@paisite.com wrote:
Hello,
I've changed the thread from "Re: Introduction, newbie confusion plus an offer" to "Turkey Questions 1-5" to start the Q&A.
Goal: Create the world's simplest web-server
My first set of questions are based on "Configuring and Using the Twisted.Web Server in "Twisted.Web Documentation."
[snip]
Questions:
1) Why did mktap write web.tap to my home directory rather than ~/twisted/www?
You didn't specify a path to a tap file to write, so it picked a reasonable-seeming default. The surprisingly named --append option lets you specify the name of the tap to write (as well as letting you add a new application to an existing tap). For example, mktap --append ~/twisted/www/web.tap web ...
[snip]
3) How can I see the source for the server created by mktap?
TAP stands for Twisted Application Pickle. Its contents are a Python pickle of a Twisted Application instance (twisted.application.service.Application) configured to do whatever you specified on the mktap command line. You can inspect it using pickle interactively: exarkun@boson:~$ python Python 2.4.1 (#2, Mar 30 2005, 21:51:10) [GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pickle >>> app = pickle.load(file('web.tap', 'rb')) >>> print app <twisted.python.components.Componentized instance at 0xb7e05b8c> >>> from twisted.application import service >>> svc = service.IService(app) >>> print svc <twisted.application.service.MultiService instance at 0xb7e11d0c> >>> list(svc) [<twisted.application.service.MultiService instance at 0xb7ae126c>] >>> list(svc)[0] <twisted.application.service.MultiService instance at 0xb7ae126c> >>> list(list(svc)[0]) [<twisted.application.internet.TCPServer instance at 0xb7a54d8c>] >>> list(list(svc)[0])[0].args (8080, <twisted.web.server.Site instance at 0xb7a5a66c>) >>> [etc] There are some other formats mktap can write, but they are generally more fragile than pickle. If you want to explore, try the `--type' argument to mktap (eg, mktap --type source web). The key here is that mktap outputs configuration, not code. Of course, if you asked "How can I view the configuration for the server generated by mktap?" I wouldn't have a better answer for you ;) Jp
lloyd@paisite.com wrote:
-- entered "mktap web --path ~/twisted/www --logfile ~/twisted/www --port 8080" into the command line -- entered "twistd -f web.tap" into command line
Oh, it's time for me to do this thing again. ( If you aren't writing an application, but just want to run a website OR If you want to host multiple applications OR You want a thing that actually serves on port 80, and not just some silly only for five minutes while you test something ) AND ( you are using Debian OR you are willing to work on porting it ) You should probably use WubWubWub. Think of it as "the apache-like thingy for twisted and nevow things". http://www.inoi.fi/open/trac/wubwubwub No pre-built debs, svn co and build yourself. Will get uploaded to Debian after minor refactoring, some day. Of course, this thing _will_ probably increase your learning curve by a bit, so I'm not suggesting all newbies do this -- I'm suggesting this thing more for "real use".
On Jul 12, 2005, at 4:06 AM, Tommi Virtanen wrote:
lloyd@paisite.com wrote:
-- entered "mktap web --path ~/twisted/www --logfile ~/twisted/www --port 8080" into the command line -- entered "twistd -f web.tap" into command line
Oh, it's time for me to do this thing again.
<snip WubWubWub instructions> Neat. I hadn't seen this before. Thanks, Tv. Donovan
Hello again, Tommi,
You should probably use WubWubWub. Think of it as "the apache-like thingy for twisted and nevow things".
http://www.inoi.fi/open/trac/wubwubwub
No pre-built debs, svn co and build yourself. Will get uploaded to Debian after minor refactoring, some day.
Of course, this thing _will_ probably increase your learning curve by a bit, so I'm not suggesting all newbies do this -- I'm suggesting this thing more for "real use".
This looks extremely cool. I'll study it and try to work one or more Turkey adventures around it. You available if I get stuck? Thanks for the suggestion, Lloyd
You should probably use WubWubWub. Think of it as "the apache-like thingy for twisted and nevow things".
http://www.inoi.fi/open/trac/wubwubwub
No pre-built debs, svn co and build yourself.
No SVN link there, so I made one up on the model of the Ldaptor's one. $ svn co svn://inoi.fi/wubwubwub/trunk svn: Can't open file '/srv/inoi.fi/repo/view/open/wubwubwub/trunk/format': Permission denied Same error when trying to access the Ldaptor SVN. -- Nicola Larosa - nico@tekNico.net Adding things just because you can leads to monstrosities like Common LISP, PL/I, Algol 68 and Perl 6. Adding features only when they add functionality (or better yet, by removing restrictions) leads to jewels like Python, Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005
Nicola Larosa wrote:
$ svn co svn://inoi.fi/wubwubwub/trunk svn: Can't open file '/srv/inoi.fi/repo/view/open/wubwubwub/trunk/format': Permission denied
Seems like I accidentally disabled anonymous SVN. Should be working now. (mental note to self: add more automatic checks)
participants (9)
-
Donovan Preston -
en.karpachov＠ospaz.ru -
James Mayfield -
Jp Calderone -
lloyd＠paisite.com -
Matt Goodall -
Nicola Larosa -
Tom Carmichael -
Tommi Virtanen