disabling redirection
Hi, I'm having problems with reverse proxies and twisted web / nevow. Which one does the redirection, in case the host or port doesn't match? How can I disable that? I want twisted to simply serve the URL it's given and not do anything fancy with the Host: header. Sorry if that sounds to harsh, I'm simply frustrated and could use some help... Regards Markus
On 09/06/2007 17:02 Markus Schiltknecht wrote:
Hi,
I'm having problems with reverse proxies and twisted web / nevow. Which one does the redirection, in case the host or port doesn't match? How can I disable that? I want twisted to simply serve the URL it's given and not do anything fancy with the Host: header.
Are you using the virtual host wrapper for correcting the hostname writing? -- Colin Alston ~ http://www.karnaugh.za.net/ Expecting one person to deal with all your problems is like praying each time you require an ambulance: You'll eventually be dead long before you get a response.
Hi, Colin Alston wrote:
Are you using the virtual host wrapper for correcting the hostname writing?
If you are talking about the VHostMonsterResource, yes, I've used that so far together with apache. But it requires to rewrite the URLs to, i.e., localhost:8080/vhost/http/my.site.com/the/real/url. With varnish, a modern and very fast reverse proxy, this is not possible (nor wanted). It simply forwards the request without adding anything to the requested URL, i.e. 'GET /the/real/url'. If that can be done with a VHostMonsterResource, too, then I'd be curious how. But as far as I understand, it allows twisted to serve multiple hosts, which I don't need. I only have exactly one host per twisted instance, so I would like to simply hardcode the host and port it's supposed to serve. But how? Regards Markus
On 09/06/2007 19:07 Markus Schiltknecht wrote:
But it requires to rewrite the URLs to, i.e., localhost:8080/vhost/http/my.site.com/the/real/url. With varnish, a modern and very fast reverse proxy, this is not possible (nor wanted). It simply forwards the request without adding anything to the requested URL, i.e. 'GET /the/real/url'.
What is the basis for using Varnish as opposed to just serving up with Twisted directly?
If that can be done with a VHostMonsterResource, too, then I'd be curious how. But as far as I understand, it allows twisted to serve multiple hosts, which I don't need. I only have exactly one host per twisted instance, so I would like to simply hardcode the host and port it's supposed to serve. But how?
Perhaps you should point out just exactly where and when you're experiencing problems with URL rewrites. I've never experienced any of these problems deploying either with a simple reverse proxy or the VHostMonstorResource. It used to be a problem specifically with Guard but these issues seem to have dissolved over the current versions. -- Colin Alston ~ http://www.karnaugh.za.net/ Expecting one person to deal with all your problems is like praying each time you require an ambulance: You'll eventually be dead long before you get a response.
Hi, Colin Alston wrote:
What is the basis for using Varnish as opposed to just serving up with Twisted directly?
Uh.. server side caching, reverse proxying... Why does that need more explanation? Why should I do that with twisted, if varnish is better at being a reverse proxy (assuming that twisted also offers a caching module, which I'm guessing)? Besides, varnish does a very good job with virtual hosts and multiple backends (web servers), given that I'm not exclusively running twisted, but other web servers as well. (And running one twisted instance per virtual website.)
Perhaps you should point out just exactly where and when you're experiencing problems with URL rewrites. I've never experienced any of these problems deploying either with a simple reverse proxy or the VHostMonstorResource. It used to be a problem specifically with Guard but these issues seem to have dissolved over the current versions.
Maybe I'm just missing an example... What I want is pretty simple: I want twisted to serve the URL which I requested in the HTTP requests top line, no matter what the Host: header says, i.e. the following two requests: GET /sample.html HTTP/1.0 Host: localhost:8080 and: GET /sample.html HTTP/1.0 Host: www.postgres-r.org ..should both deliver the same sample.html file. And *not* a redirect to whatever twisted thinks is it's hostname and port. If twisted could simply ignore the 'Host' field, that would be fine. If I could tell it, what Hosts it should accept (instead of only accepting 'localhost:8080') and what not (i.e. it should probably redirect 'postgres-r.org' to 'www.postgres-r.org'), that would be extra super! Maybe, someone could simply point me to the python file where twisted does the hostname and port checking and the redirection? That would already help... Regards Markus
On Sun, 10 Jun 2007 17:32:47 +0200, Markus Schiltknecht
Hi,
Colin Alston wrote:
What is the basis for using Varnish as opposed to just serving up with Twisted directly?
Uh.. server side caching, reverse proxying... Why does that need more explanation? Why should I do that with twisted, if varnish is better at being a reverse proxy (assuming that twisted also offers a caching module, which I'm guessing)?
Besides, varnish does a very good job with virtual hosts and multiple backends (web servers), given that I'm not exclusively running twisted, but other web servers as well. (And running one twisted instance per virtual website.)
Perhaps you should point out just exactly where and when you're experiencing problems with URL rewrites. I've never experienced any of these problems deploying either with a simple reverse proxy or the VHostMonstorResource. It used to be a problem specifically with Guard but these issues seem to have dissolved over the current versions.
Maybe I'm just missing an example...
What I want is pretty simple: I want twisted to serve the URL which I requested in the HTTP requests top line, no matter what the Host: header says, i.e. the following two requests:
GET /sample.html HTTP/1.0 Host: localhost:8080
and:
GET /sample.html HTTP/1.0 Host: www.postgres-r.org
..should both deliver the same sample.html file. And *not* a redirect to whatever twisted thinks is it's hostname and port.
If twisted could simply ignore the 'Host' field, that would be fine. If I could tell it, what Hosts it should accept (instead of only accepting 'localhost:8080') and what not (i.e. it should probably redirect 'postgres-r.org' to 'www.postgres-r.org'), that would be extra super!
Maybe, someone could simply point me to the python file where twisted does the hostname and port checking and the redirection? That would already help...
You're going about this wrong, unfortunately. "Twisted" won't issue a redirect based on the Host header. Even "twisted.web" won't do that. Some resource provided by twisted.web (or nevow, I forget what you're using) might, but only if you put it into your resource hierarchy. Perhaps you could elaborate on how and of what your Site is constructed? A minimal code example would be ideal, since it communicates exactly the problem you're having, instead of relying on the imperfect ability of other people to translate a prose description of a program back into the same program. Jean-Paul
Hi, Jean-Paul Calderone wrote:
You're going about this wrong, unfortunately. "Twisted" won't issue a redirect based on the Host header. Even "twisted.web" won't do that.
Okay, good to know. (And good it is that way)
Some resource provided by twisted.web (or nevow, I forget what you're using) might, but only if you put it into your resource hierarchy. Perhaps you could elaborate on how and of what your Site is constructed? A minimal code example would be ideal, since it communicates exactly the problem you're having, instead of relying on the imperfect ability of other people to translate a prose description of a program back into the same program.
After the frustrating experience with various caches (server and browser side) interrupting and adding to the confusion, I didn't feel like compiling a test case, sorry. Today, thanks to you motivating mail, I've tried again and tcpdump'ed the h*** out of it! I think I've finally tracked down what's bugging me: it's the addSlash = True forwarder (presumably in the nevow code). (Sorry if that's the wrong list for nevow related stuff - is there a dedicated nevow mailing list?) Here's how to reproduce my problem: ##### sample.tac ##### from nevow import rend, loaders from nevow import tags as T from nevow.appserver import NevowSite from twisted.application import service, internet class Child(rend.Page): addSlash = True docFactory = loaders.stan(T.html[ T.p[ "Hello, child!" ] ]) class Site(rend.Page): docFactory = loaders.stan(T.html[ T.p[ "Hello, world!" ] ]) child_test = Child() application = service.Application('sample') internet.TCPServer( 8081, NevowSite(Site(), logPath="web.log") ).setServiceParent(application) ##### telnet session to that server ##### markus@argodan:/home/markus# telnet localhost 8081 Trying 127.0.0.1... Connected to argodan. Escape character is '^]'. GET / HTTP/1.1 Host: somewhere.com HTTP/1.1 200 OK Transfer-encoding: chunked Date: Mon, 11 Jun 2007 07:59:37 GMT Content-type: text/html; charset=UTF-8 Server: TwistedWeb/2.4.0 21 <html><p>Hello, world!</p></html> 0 GET /test HTTP/1.1 Host: somewhere.com HTTP/1.1 302 Found Transfer-encoding: chunked Date: Mon, 11 Jun 2007 07:59:46 GMT Content-type: text/html; charset=UTF-8 Location: http://somewhere.com:8081/test/ <--- I'VE STUMBLED OVER Server: TwistedWeb/2.4.0 THIS REDIRECT HERE 0 GET /test/ HTTP/1.1 Host: somewhere.com HTTP/1.1 200 OK Transfer-encoding: chunked Date: Mon, 11 Jun 2007 07:59:55 GMT Content-type: text/html; charset=UTF-8 Server: TwistedWeb/2.4.0 21 <html><p>Hello, child!</p></html> 0 So, the question now is: how do I tell nevow where it should redirect to in case of 'addSlash = True'? I don't want it to redirect to the internal port 8081. [ Why does it use the Host: header field, but not the corresponding port? Maybe this has already been fixed and I only need an update? Or does this have a good reason for being as it is? ] Thanks for you help. Regards Markus
Hi, Some time ago, I wrote:
Here's how to reproduce my problem: ...
And now, here comes how I've solved it. My solution forces nevow to think, that the request has been made for a given host and port. If that's not really the case, nevow will serve the Resource anyway - that's no different from not using the ForcedHost wrapper. Only in case nevow wants to issue a 301 redirect (due to addSlash=True), the forced host (and port) apply. Nevow will redirect to that host:port. class ForcedHost(rend.Page): def __init__(self, res, host, port=80): self.wrappedResource = res self.host = host self.port = port def locateChild(self, ctx, segments): request = inevow.IRequest(ctx) request.setHost(self.host, self.port) return self.wrappedResource.locateChild(ctx, segments) siteRoot = ForcedHost(MyNevowResource, 'www.my-forced-site.org') Does anybody see any problems with this approach? Regards Markus
participants (3)
-
Colin Alston
-
Jean-Paul Calderone
-
Markus Schiltknecht