Thomas Weholt firstname.lastname@example.org writes:
A few users connect to the main server, requesting files on the subnet web-servers. The main server reads data from several subnet servers and writes the data back to the requesting clients. How can I do this in twisted, without blocking, and handle several clients? We're not talking hardcore P2P here with thousands of clients, most likely 2-5 concurrent users, 10 at the most.
I haven't had an opportunity to use it myself yet, but there is a twisted.web.spread module that may work since all of your servers are using Twisted.
On the subnet servers, wrap your site object in the ResourcePublisher object (a pb.Root subclass) and set that up to listen on an appropriate port.
On the main server, for each resource root that you want proxied out to a remote server, insert an appropriate child resource using ResourceSubscription.
A PB link used to transmit the requests between the two machines, which means that all of the server render() calls to be proxied are handled in a deferred fashion.
I expect there may be a way to interconnect your server into a client HTTP class to proxy to the subset servers using a more traditional web request, but given that you are Twisted throughout, probably no reason not to go ahead and use the PB approach.
It does look like these classes still use the older pb.getObjectAt approach for the connection rather than the newer getRootObject, but it should still work. If it doesn't directly suit what you want, it it may at least give you an idea for your own approach (since neither of those classes is overly complex).