[Twisted-Python] Server with several outgoing connections

Hi! I'm new to Twisted and I have tried to read and understand the intricate sides of it. And have found as so many other that the learning curve pretty steep, especially I guess if one comes from being a C programmer :-) Anyway, this is my problem: I'm building a system that consists of a number of nodes, each node acting as a receiver and transmitter of messages. The protocol used between the nodes might be a message passing system like xmpp or something else. The number of nodes are dynamic and might change at any time. Plus there is no way by which you can know when the system is started how many nodes there will be. Also nodes might receive messages from any number of nodes and be expected to transmit the messages to several nodes. Writing the 'server' part of a node is easy, plenty of examples of that exists on the net. Writing the 'client' side is not so easy and here I guess my C legacy is hampering me. So I'd like someone to give me, or point me, to an example on how to dynamically create new connections to receivers. I first though these connections should be shortlived that is 'open the connection, send the message, receive the ACK and close the connection' but now I think it might be wiser to keep the connections as long as possible. That is, until that receiver disappears. Another side of this is that the influx of messages might be higher than the possible output, that is each node has to keep queues for messages not yet sent and messages sent but not ack'ed. It might also happen that recivers of messages are not accessible in which case the message has to be queue until the receiver pops up again. The system is not allowed to drop messages. Note, that each incomming message might have several receivers, so one incomming message might end up in the sendqueue of several receivers. And here in lies one of the problems, I guess I have to keep a 'centrally' managed queue, that should then be notified on the change of status of a message on route to a recevier. How should I implement this ? --Roland

On Thu, 4 Nov 2004 10:37:53 +0100, Roland Hedberg <roland.hedberg@adm.umu.se> wrote:
Hi!
Hello.
I'm new to Twisted and I have tried to read and understand the intricate sides of it. And have found as so many other that the learning curve pretty steep, especially I guess if one comes from being a C programmer :-)
Anyway, this is my problem:
I'm building a system that consists of a number of nodes, each node acting as a receiver and transmitter of messages. The protocol used between the nodes might be a message passing system like xmpp or something else. The number of nodes are dynamic and might change at any time. Plus there is no way by which you can know when the system is started how many nodes there will be. Also nodes might receive messages from any number of nodes and be expected to transmit the messages to several nodes.
Writing the 'server' part of a node is easy, plenty of examples of that exists on the net.
Writing the 'client' side is not so easy and here I guess my C legacy is hampering me.
So I'd like someone to give me, or point me, to an example on how to dynamically create new connections to receivers.
I don't see the need for anything special. The reactor.connectTCP function is probably what you want.
I first though these connections should be shortlived that is 'open the connection, send the message, receive the ACK and close the connection' but now I think it might be wiser to keep the connections as long as possible. That is, until that receiver disappears.
Just call transport.loseConnection() when you want it to go away.
Another side of this is that the influx of messages might be higher than the possible output, that is each node has to keep queues for messages not yet sent and messages sent but not ack'ed. It might also happen that recivers of messages are not accessible in which case the message has to be queue until the receiver pops up again. The system is not allowed to drop messages. Note, that each incomming message might have several receivers, so one incomming message might end up in the sendqueue of several receivers.
...
And here in lies one of the problems, I guess I have to keep a 'centrally' managed queue, that should then be notified on the change of status of a message on route to a recevier. How should I implement this ?
*shrug*, depends on the application. Do you want central management? Do you really need it? You really need to ask some more specific questions before anyone can give you good answers. -Eric

I'm using twisted and nevow and would like to implement a log in system for my site. The view of the site will differ based on the credentials a user supplies. I would also (obviously) like to prevent any circumvention of the log in via deep linking or any other means. I vaguely remember a while back some discussion of cred/newcred, but I do not know what classes and modules correspond to each system. I also see a nevow guard class that may be useful, or may be old, with a mysterious comment: Resource protection for Nevow. If you wish to use twisted.cred to protect your Nevow application, you are probably most interested in L{SessionWrapper}. """ What modules and classes should I be looking at? Are there any examples out there? Is the how-to on the website (http://twistedmatrix.com/documents/current/howto/cred) up to date? Thanks, Alex

On Nov 5, 2004, at 10:59 AM, Alexander May wrote:
I'm using twisted and nevow and would like to implement a log in system for my site. The view of the site will differ based on the credentials a user supplies. I would also (obviously) like to prevent any circumvention of the log in via deep linking or any other means.
I vaguely remember a while back some discussion of cred/newcred, but I do not know what classes and modules correspond to each system. I also see a nevow guard class that may be useful, or may be old, with a mysterious comment:
Resource protection for Nevow. If you wish to use twisted.cred to protect your Nevow application, you are probably most interested in L{SessionWrapper}. """
What modules and classes should I be looking at? Are there any examples out there? Is the how-to on the website (http://twistedmatrix.com/documents/current/howto/cred) up to date?
There are several examples of using nevow.guard and implementing a simple cred realm in the nevow examples directory. If you need more help, it is more appropriate to ask on the twisted-web mailing list, where you are more likely to get an answer: http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web Another good place to get help is the #twisted.web channel on irc.freenode.net. dp

I did up a few howtos in my blog a while back: http://blog.vrplumber.com/356 http://blog.vrplumber.com/358 http://blog.vrplumber.com/371 which covers the "official" way to protect a Nevow site (as gleaned from IRC conversations with Nevow's creators). I think Mary was planning on doing up official docs for the process at some point. HTH, Mike Alexander May wrote: ...
What modules and classes should I be looking at? Are there any examples out there? Is the how-to on the website (http://twistedmatrix.com/documents/current/howto/cred) up to date?
... ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com

Thanks Mike and Donovan. The examples were very helpful. Two questions though: 1) If the user types in something like mysite.com/foo/bar and is not yet logged in, they get the "Sorry, but I couldn't find the object you requested." page. I would like them to get the log in page, and then be redirected to the mysite.com/foo/bar afterwards. Can I do this? 2) How do I replace the "Sorry, but I couldn't find the object you requested." with my own page? Thanks, Alex (cross posted to twisted web group) -----Original Message----- From: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of Mike C. Fletcher Sent: Friday, November 05, 2004 2:55 PM To: Twisted general discussion Subject: Re: [Twisted-Python] Log in - state of the art? I did up a few howtos in my blog a while back: http://blog.vrplumber.com/356 http://blog.vrplumber.com/358 http://blog.vrplumber.com/371 which covers the "official" way to protect a Nevow site (as gleaned from IRC conversations with Nevow's creators). I think Mary was planning on doing up official docs for the process at some point. HTH, Mike Alexander May wrote: ...
What modules and classes should I be looking at? Are there any examples out there? Is the how-to on the website (http://twistedmatrix.com/documents/current/howto/cred) up to date?
... ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Minor correction for question 1. My specific case is: Mysite.com/abcd.html, not Mysite.com/foo/bar -----Original Message----- From: twisted-web-bounces@twistedmatrix.com [mailto:twisted-web-bounces@twistedmatrix.com] On Behalf Of Alexander May Sent: Friday, November 05, 2004 5:00 PM To: 'Twisted general discussion'; twisted-web@twistedmatrix.com Subject: [Twisted-web] RE: [Twisted-Python] Log in - state of the art? Thanks Mike and Donovan. The examples were very helpful. Two questions though: 1) If the user types in something like mysite.com/foo/bar and is not yet logged in, they get the "Sorry, but I couldn't find the object you requested." page. I would like them to get the log in page, and then be redirected to the mysite.com/foo/bar afterwards. Can I do this? 2) How do I replace the "Sorry, but I couldn't find the object you requested." with my own page? Thanks, Alex (cross posted to twisted web group) -----Original Message----- From: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of Mike C. Fletcher Sent: Friday, November 05, 2004 2:55 PM To: Twisted general discussion Subject: Re: [Twisted-Python] Log in - state of the art? I did up a few howtos in my blog a while back: http://blog.vrplumber.com/356 http://blog.vrplumber.com/358 http://blog.vrplumber.com/371 which covers the "official" way to protect a Nevow site (as gleaned from IRC conversations with Nevow's creators). I think Mary was planning on doing up official docs for the process at some point. HTH, Mike Alexander May wrote: ...
What modules and classes should I be looking at? Are there any examples out there? Is the how-to on the website (http://twistedmatrix.com/documents/current/howto/cred) up to date?
... ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

Alexander May wrote: ...
1) If the user types in something like mysite.com/foo/bar and is not yet logged in, they get the "Sorry, but I couldn't find the object you requested." page. I would like them to get the log in page, and then be redirected to the mysite.com/foo/bar afterwards. Can I do this?
Yes, you want to make the __login__ page precede the rest of the URL, so in my case, for the loginForm's locationChild method I do this: def locateChild(self, ctx, segments): """Locate child, return as inevow.IResource""" if segments: if segments[0] == 'style': return self.style, segments[1:] elif segments[0] == 'images': return self.images, segments[1:] inevow.IRequest(ctx).args[ 'finalURL'] = "/".join(segments) return self, () then in the form itself I access finalURL and make the address of the form reflect the final destination: <form tal:attributes="action python: '/%s/%s'%('__login__',REQUEST.form.get('finalURL',''))" method="get"> This doesn't, however, deal with re-encoding form values, so abcd.html?this=that still won't work. Better approaches are likely available with a little more spelunking through the authorisation code...
2) How do I replace the "Sorry, but I couldn't find the object you requested." with my own page?
Haven't done this myself, from what I can see it would require some hacking inside Nevow: from appserver.py if newres is None: from nevow.rend import FourOhFour return context.PageContext(tag=FourOhFour(), parent=pageContext) where FourOhFour is just an IResource, but short of actually assigning to nevow.rend.FourOhFour I don't see a way to override it. Good luck, Mike ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com

Alle 01:26, sabato 6 novembre 2004, Mike C. Fletcher ha scritto:
Alexander May wrote: ... ....
if newres is None: from nevow.rend import FourOhFour return context.PageContext(tag=FourOhFour(), parent=pageContext)
where FourOhFour is just an IResource, but short of actually assigning to nevow.rend.FourOhFour I don't see a way to override it.
Good luck, Mike
with nevow 0.2, I used to do the following in the root resource: class Root(rend.Page): def __init__(self, original): super(Page, self).__init__(original) self.remember(self,inevow.ICanHandleNotFound) def renderHTTP_notFound(self, request): return MyFourOFourPage() I think/hope this still works, but haven't tried it recently
________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Pradu Linux webmaster & sysadmin pradu@pradu.it

On Fri, Nov 05, 2004 at 02:55:06PM -0500, Mike C. Fletcher wrote:
I did up a few howtos in my blog a while back:
you example to create the keys was helpful thanks. On the client side I would like to stress encryptions is worthless if we can't verify the certificate on the other side and there seem not much power into verifying the certificate in twisted today (if we can't verify the certificate on the other side a man in the middle may see everything and forward the packet to the real server). Everybody writing a client should always verify the certificate somehow otherwise encryption is worthless. So the server side is fine, but the client side could be more oriented towards verification of the certificate. I'm copying the cacert.pem file (generated as in your blog) into the client package (the certificate is basically the public key, so I can make it public) and then the client source will be like this: def verify_certificate(conn, cert, errno, depth, retcode): return crypto.dump_certificate(crypto.FILETYPE_PEM, cert) == server_certificate class client_context_factory(ssl.ClientContextFactory): def getContext(self): ctx = SSL.Context(self.method) ctx.set_verify(SSL.VERIFY_PEER, verify_certificate) return ctx def main(): factory = EchoClientFactory() global server_certificate server_certificate = file("privkey.pem").read() reactor.connectSSL('localhost', 8000, factory, client_context_factory()) reactor.run() this works fine and the connection is aborted if the certificate is wrong, but I can't intercept the exception to print a meaningful message to the user. This is not a big problem though it'll prevent me to ask the user to autosubmit the stack trace to the server for debugging purposes. the problem is described here too: http://twistedmatrix.com/pipermail/twisted-python/2003-December/006803.html I believe SSL is better suited than the ssh protocol for my needs (I only need to wrap some encryption over a socket and I want to write the lowest amount of code as possible). With a 8192 bits key I get around 4 packets sent all under 1400 per-packet (so they should fit most MTUs). Size of the packets is 100 (fixed), 1348, 1228, 1108. A 2048byte key would get away with just 3 packets (the 1348+1228 would shink to a single packet), but I prefer stronger crypto despite the 1 more pack per connection. I like the idea to multiplex the connection heavily on the client side, so each client has only 1 channel open with the server. This should help to keep the encrytpion cost (cpu and network) down too. I'm a bit scared that python will become a bottleneck if I multiplex inside python instead of in the tcp stack, but I expect in the short term to be network bound even in the server side (the client side is obviously always network bound these days). At least the server side algorithm is greatly scalable (there's only 1 very cpu-light operation that is not scalable and has to reside on a single machine and in the future I may need to rewrite that single but in C definitely with epoll), the multiplex and the polling is scalable (though I will need epoll eventually), so I can throw an hardware farm at the problem if I hit a cpu bottleneck with too many clients connected. My spare time is more worthy than hardware ;). I'm optimistic I will not have to rewrite the whole server in C later on ;) PS. I didn't use the days parameter while generating the certificate, I assume that means "unlimited" time.

On Sat, 2004-11-06 at 18:39 +0100, Andrea Arcangeli wrote:
this works fine and the connection is aborted if the certificate is wrong, but I can't intercept the exception to print a meaningful message to the user. This is not a big problem though it'll prevent me to ask the user to autosubmit the stack trace to the server for debugging purposes.
Please add an issue in the bug/feature tracker about this (http://twistedmatrix.com/bugs/) and put me on the nosy list. If it's not in the bug tracker it's not likely to get added cause we'll forget.

On Sat, Nov 06, 2004 at 12:48:12PM -0500, Itamar Shtull-Trauring wrote:
Please add an issue in the bug/feature tracker about this (http://twistedmatrix.com/bugs/) and put me on the nosy list. If it's not in the bug tracker it's not likely to get added cause we'll forget.
ok thanks.

On Sat, 06 Nov 2004 12:48:12 -0500, Itamar Shtull-Trauring <itamar@itamarst.org> wrote:
On Sat, 2004-11-06 at 18:39 +0100, Andrea Arcangeli wrote:
this works fine and the connection is aborted if the certificate is wrong, but I can't intercept the exception to print a meaningful message to the user. This is not a big problem though it'll prevent me to ask the user to autosubmit the stack trace to the server for debugging purposes.
Please add an issue in the bug/feature tracker about this (http://twistedmatrix.com/bugs/) and put me on the nosy list. If it's not in the bug tracker it's not likely to get added cause we'll forget.
There's already a bug in the tracker, and I added some code along these lines yesterday. However, after some discussion with James Knight, I don't think the set_verify OpenSSL API is of any relevance to this use case. Andrea would probably be better off verifying the client certificate at a later time. The discussion of exactly where really belongs on the twisted-web mailing list, but as far as the SSL details are concerned, rejecting the certificate at the handshake verification time causes the handshake to fail; while it is possible that this is a non-fatal error, many implementations will simply drop the connection anyway. When performing an application-level certificate verification subsequently, a secure communication channel has been established and can be used to communicate the error to the user as, say, a web page informing them of the invalidity of their certificate. The drawback of this approach is that it allows attackers to charge you the cost of a full SSL handshake and a bit of encrypted traffic. The advantages seem to far outweigh this. A further problem with the OpenSSL verify callback is that it is synchronous and called in the IO thread. For any non-trivial application, this is likely to end up blocking the reactor in unacceptable ways. Jp

On Sat, Nov 06, 2004 at 06:04:43PM +0000, exarkun@divmod.com wrote:
There's already a bug in the tracker, and I added some code along these lines yesterday.
However, after some discussion with James Knight, I don't think the set_verify OpenSSL API is of any relevance to this use case. Andrea would probably be better off verifying the client certificate at a later time. The discussion of exactly where really belongs on the
if you can tell me where should I check it that's fine with me.
twisted-web mailing list, but as far as the SSL details are
I'm not going to do any web sort of thing, I'm using ssl only to wrap encryption around a stocket. It looked the fastest way. the protocol running on top of ssl is a not standard one. This is why I posted here and not to twisted-web (I'm not even subscribed there).
concerned, rejecting the certificate at the handshake verification time causes the handshake to fail; while it is possible that this is a non-fatal error, many implementations will simply drop the connection anyway.
When performing an application-level certificate verification subsequently, a secure communication channel has been established and can be used to communicate the error to the user as, say, a web page informing them of the invalidity of their certificate. The
this doesn't look possible, the server side that should emit the graceful webpage is a malicious attack that would rather show you the true faked website instead of a warning. Only the client can notice the error and the graceful message I was going to add was infact in the client (as soon as I notice the server is the wrong one and it doesn't match the client certificate stored on the client harddisk, I will immediatly stop talking with the server).
drawback of this approach is that it allows attackers to charge you the cost of a full SSL handshake and a bit of encrypted traffic. The advantages seem to far outweigh this.
full ssl handshake during an attack is fine ;).
A further problem with the OpenSSL verify callback is that it is synchronous and called in the IO thread. For any non-trivial application, this is likely to end up blocking the reactor in unacceptable ways.
good point. luckily this is not a problem at all for my case, since it's only invoked in the client and when the client startup, it won't be asynchronous anyways, since python also need to read the *.py sources from disk ;) the server truly needs to be always async, and the server won't use the verify callback (that can only be useful in the client). Anyways I'm all hears if you've a better solution to do the verify. I don't claim my solution to be good, but this was the only way I could make it work so far (and it's pretty much undocumented, that's why I said it may be appropriate to put more focus on the verification of the certificate as a more official API in twisted, because without any verification the ssl protocol is weak). Please let me know if I should subscribe to twisted-web or if it's ok to discuss the strict SSL issues here. thanks for the help!

On Sat, 6 Nov 2004 19:23:01 +0100, Andrea Arcangeli <andrea@cpushare.com> wrote:
On Sat, Nov 06, 2004 at 06:04:43PM +0000, exarkun@divmod.com wrote:
There's already a bug in the tracker, and I added some code along these lines yesterday.
However, after some discussion with James Knight, I don't think the set_verify OpenSSL API is of any relevance to this use case. Andrea would probably be better off verifying the client certificate at a later time. The discussion of exactly where really belongs on the
if you can tell me where should I check it that's fine with me.
twisted-web mailing list, but as far as the SSL details are
I'm not going to do any web sort of thing, I'm using ssl only to wrap encryption around a stocket. It looked the fastest way. the protocol running on top of ssl is a not standard one. This is why I posted here and not to twisted-web (I'm not even subscribed there).
Ah, sorry. I confused you with the poster who started this thread and was doing web-related things, and so also misinterpreted the bug/feature you were talking about. After re-reading your first post, I see what you mean (and it is indeed something that needs to be fixed, as itamar indicated). Jp

You might also want to consider the design of your application in case it's not the best way of solving the problem using Twisted (for example if all the nodes run on the same computer ....) On Fri, 05 Nov 2004 10:39:09 -0600, Eric Mangold <teratorn@world-net.net> wrote:
On Thu, 4 Nov 2004 10:37:53 +0100, Roland Hedberg <roland.hedberg@adm.umu.se> wrote:
Hi!
Hello.
I'm new to Twisted and I have tried to read and understand the intricate sides of it. And have found as so many other that the learning curve pretty steep, especially I guess if one comes from being a C programmer :-)
Anyway, this is my problem:
I'm building a system that consists of a number of nodes, each node acting as a receiver and transmitter of messages. The protocol used between the nodes might be a message passing system like xmpp or something else. The number of nodes are dynamic and might change at any time. Plus there is no way by which you can know when the system is started how many nodes there will be. Also nodes might receive messages from any number of nodes and be expected to transmit the messages to several nodes.
Writing the 'server' part of a node is easy, plenty of examples of that exists on the net.
Writing the 'client' side is not so easy and here I guess my C legacy is hampering me.
So I'd like someone to give me, or point me, to an example on how to dynamically create new connections to receivers.
I don't see the need for anything special. The reactor.connectTCP function is probably what you want.
I first though these connections should be shortlived that is 'open the connection, send the message, receive the ACK and close the connection' but now I think it might be wiser to keep the connections as long as possible. That is, until that receiver disappears.
Just call transport.loseConnection() when you want it to go away.
Another side of this is that the influx of messages might be higher than the possible output, that is each node has to keep queues for messages not yet sent and messages sent but not ack'ed. It might also happen that recivers of messages are not accessible in which case the message has to be queue until the receiver pops up again. The system is not allowed to drop messages. Note, that each incomming message might have several receivers, so one incomming message might end up in the sendqueue of several receivers.
...
And here in lies one of the problems, I guess I have to keep a 'centrally' managed queue, that should then be notified on the change of status of a message on route to a recevier. How should I implement this ?
*shrug*, depends on the application. Do you want central management? Do you really need it?
You really need to ask some more specific questions before anyone can give you good answers.
-Eric
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (10)
-
Alessandro Praduroux
-
Alexander May
-
Andrea Arcangeli
-
Donovan Preston
-
Eric Mangold
-
exarkun@divmod.com
-
Itamar Shtull-Trauring
-
Mike C. Fletcher
-
Roland Hedberg
-
Seun Osewa