[Twisted-Python] Multiple clients
Hy, Can somebody send me a small example how to use multiple clients at same time? For example to connect to 3 different IMAP server. I need this because I'm working on an email client engine. I can connect to POP/IMAP/SMTP server to download and send messages, but I have created those methods separated, and each time when I check for new messages or send a message I use reactor.run() and reactor.stop(). But calling it multiple times is a problem. So I need something: - when I press button1, the application checks for new messages on POP server. - when I press button2, the application checks for new messages on IMAP server. I know how to check for messages on IMAP/POP server, but I don't know how to do it periodically with twisted... I hope you understand what I'm trying to explain :}} Thanks in advance Szabolcs Balogh
On 10:36 am, bszabolcs@gmail.com wrote:
Hy,
Can somebody send me a small example how to use multiple clients at same time? For example to connect to 3 different IMAP server. I need this because I'm working on an email client engine. I can connect to POP/IMAP/SMTP server to download and send messages, but I have created those methods separated, and each time when I check for new messages or send a message I use reactor.run() and reactor.stop(). But calling it multiple times is a problem.
Why?
So I need something: - when I press button1, the application checks for new messages on POP server. - when I press button2, the application checks for new messages on IMAP server.
How about: def onbutton1(self): checkPOP() def onbutton2(self): checkIMAP()
I know how to check for messages on IMAP/POP server, but I don't know how to do it periodically with twisted...
Periodically as in "once every N seconds"? Check out reactor.callLater or twisted.internet.task.LoopingCall. Jean-Paul
I have used the following code, which is working until I called it multiple time: class ImapProtocol(twisted.mail.imap4.IMAP4Client): ... implementation ... class ImapFactory(twisted.internet.protocol.ClientFactory): def __init__(self, account): self.account = account self.deferred = twisted.internet.defer.Deferred() self.protocol = ImapProtocol def clientConnectionFailed(self, connection, reason): """ Called when the client has failed to connect """ self.deferred.errback(reason) class ImapClient(gobject.GObject): def __init__(self, account): super(ImapClient, self).__init__() self.account = account self.host = self.account.get_server_address() self.port = self.account.get_server_port() self.ssl = self.account.get_use_ssl() def got_mailbox_list(self, result): twisted.internet.reactor.stop() def handle_error(self, error): twisted.internet.reactor.stop() def get_mailbox_list(self): """ Get the list of existing mailboxes. """ imapfactory = ImapFactory(self.account, self.LIST_MAILBOX) imapfactory.deferred.addCallback(self.got_mailbox_list) imapfactory.deferred.addErrback(self.handle_error) if self.ssl: # Using SSL for encrypted connections contextfactory = twisted.internet.ssl.ClientContextFactory() twisted.internet.reactor.connectSSL(self.host, self.port, imapfactory, contextfactory, 4) else: # Server doesn't require SSL twisted.internet.reactor.connectTCP(self.host, self.port, imapfactory, 4) twisted.internet.reactor.run() I call it in the following way: imap_client = imapclient.ImapClient(account) imap_client.get_mailbox_list() If I call it only once, it executes successfully, but if I call it twice it get blocks and never finishes... So how can I start the reactor and use the connectSSL/connectTCP methods multiple times? How to add new clients to an existing reactor? On Friday 18 June 2010 16:01:02 exarkun@twistedmatrix.com wrote:
On 10:36 am, bszabolcs@gmail.com wrote:
Hy,
Can somebody send me a small example how to use multiple clients at same time? For example to connect to 3 different IMAP server. I need this because I'm working on an email client engine. I can connect to POP/IMAP/SMTP server to download and send messages, but I have created those methods separated, and each time when I check for new messages or send a message I use reactor.run() and reactor.stop(). But calling it multiple times is a problem.
Why?
So I need something: - when I press button1, the application checks for new messages on POP server. - when I press button2, the application checks for new messages on IMAP server.
How about:
def onbutton1(self): checkPOP()
def onbutton2(self): checkIMAP()
I know how to check for messages on IMAP/POP server, but I don't know how to do it periodically with twisted...
Periodically as in "once every N seconds"? Check out reactor.callLater or twisted.internet.task.LoopingCall.
Jean-Paul
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Friday 18 June 2010 17:49:27 Phil Mayers wrote:
On 18/06/10 15:34, Szabolcs Balogh wrote:
def get_mailbox_list(self): twisted.internet.reactor.run()
The reactor isn't restartable. You can only start it once.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Than how can I start checking for messages for different accounts (When I have finished checking the messages for the first account to start checking for the next account, and so on...)?
Couldn't you use callLater to fire the first account check every N interval and then just attach the subsequent accounts for that interval as callbacks to the first check inside your callLater func? -J On Fri, Jun 18, 2010 at 9:25 AM, Szabolcs Balogh <bszabolcs@gmail.com> wrote:
On Friday 18 June 2010 17:49:27 Phil Mayers wrote:
On 18/06/10 15:34, Szabolcs Balogh wrote:
def get_mailbox_list(self): twisted.internet.reactor.run()
The reactor isn't restartable. You can only start it once.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Than how can I start checking for messages for different accounts (When I have finished checking the messages for the first account to start checking for the next account, and so on...)?
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Jun 18, 2010, at 8:25 AM, Szabolcs Balogh wrote:
Than how can I start checking for messages for different accounts (When I have finished checking the messages for the first account to start checking for the next account, and so on...)?
The whole *point* of the "reactor" is to allow multiple connections (and timed calls) to be serviced simultaneously. When you have "finished checking messages" for the first account, presumably there is a callback of some kind; most likely 'connectionLost()'. In connectionLost, you can just call connectTCP again with a protocol factory pointed at the second account. This is actually a FAQ: <http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoImakeTwiste...> Hopefully this helps. This is a hard question to answer in a way where the next asker will actually find the answer, since the answer is really "just call a method", and the asker needs to understand that they can call any of these methods at any time. If you have any idea as to how the FAQ could be improved so that you might have found your answer while searching around, please let us know :). For example: the way you phrased your requirement here, "When I have done X, start doing X", even *sounds* like an event-driven program - 'def iHaveFinishedChecking(self): next.startChecking()' - so you were clearly on the right track. Why did you think you needed to run the reactor again in order to accomplish this?
On Friday 18 June 2010 19:24:54 Glyph Lefkowitz wrote:
On Jun 18, 2010, at 8:25 AM, Szabolcs Balogh wrote:
Than how can I start checking for messages for different accounts (When I have finished checking the messages for the first account to start checking for the next account, and so on...)?
The whole *point* of the "reactor" is to allow multiple connections (and timed calls) to be serviced simultaneously.
When you have "finished checking messages" for the first account, presumably there is a callback of some kind; most likely 'connectionLost()'. In connectionLost, you can just call connectTCP again with a protocol factory pointed at the second account.
This is actually a FAQ:
<http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoImakeTwis tedtalktomultipleclientsconnecttomultipleservers>
Hopefully this helps.
This is a hard question to answer in a way where the next asker will actually find the answer, since the answer is really "just call a method", and the asker needs to understand that they can call any of these methods at any time. If you have any idea as to how the FAQ could be improved so that you might have found your answer while searching around, please let us know :). For example: the way you phrased your requirement here, "When I have done X, start doing X", even *sounds* like an event-driven program - 'def iHaveFinishedChecking(self): next.startChecking()' - so you were clearly on the right track. Why did you think you needed to run the reactor again in order to accomplish this?
Using: twisted.internet.reactor.connectTCP(host, port, imapfactory, 4) twisted.internet.reactor.run() it works. But if I try: twisted.internet.reactor.run() twisted.internet.reactor.connectTCP(host, port, imapfactory, 4) it stops at run() and doesn't executes the connectTCP method. This is why I thought that first I have to execute connectTCP and then reactor.run()
Couldn't you use callLater to fire the first account check every N interval and then just attach the subsequent accounts for that interval as callbacks to the first check inside your callLater func?
I can't use the callLater, because I'm working on the engine part of an email client. So I need a "get_messages(imap_account)" method, which can be called from the GUI synchron or asynchron (pressed the check now button by the user) and checks for new messages for the account specified. So it needs to be as generally as possible and automatically doesn't do anything.
On Friday 18 June 2010 19:24:54 Glyph Lefkowitz wrote:
This is actually a FAQ:
<http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoImakeTwis tedtalktomultipleclientsconnecttomultipleservers>
"Sometimes people ask this question when they write a function that calls connectTCP, then reactor.run(). You don't usually need to call reactor.run() yourself; let twistd do it. If you do need to call it yourself, call it just once after your initial setup. When reactor.run() exits, your program should too." It sounds good, but if I don't call reactor.run() after connectSSL doesn't happens anything... If I call reactor.run() after connectSSL, it works properly...
It sounds good, but if I don't call reactor.run() after connectSSL doesn't happens anything... If I call reactor.run() after connectSSL, it works properly...
There are two types of client connections: 1. Connections you open when you first run the program. Call these before reactor.run() is called. 2. Connections that happen as result of events. Events are all dispatched by the reactor, so e.g. gui click or data received over network connection will result in functions being called by the reactor. These functions can also open new connections. The key here is to distinguish between the order of the code in the text file, and the order the code runs in. In the following example, the connectTCP will happen *after* reactor.run(): # this will get called 10 seconds after reactor.run(): def happensLater(): reactor.connectTCP(....) reactor.callLater(10, happensLater) reactor.run()
On Friday 18 June 2010 20:25:50 Itamar Turner-Trauring wrote:
It sounds good, but if I don't call reactor.run() after connectSSL doesn't happens anything... If I call reactor.run() after connectSSL, it works properly...
There are two types of client connections:
1. Connections you open when you first run the program. Call these before reactor.run() is called.
2. Connections that happen as result of events. Events are all dispatched by the reactor, so e.g. gui click or data received over network connection will result in functions being called by the reactor. These functions can also open new connections.
The key here is to distinguish between the order of the code in the text file, and the order the code runs in. In the following example, the connectTCP will happen *after* reactor.run():
# this will get called 10 seconds after reactor.run(): def happensLater(): reactor.connectTCP(....)
reactor.callLater(10, happensLater) reactor.run()
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
It is possible to: - start the reactor when I initialize my program (eg. in main) - call connectTCP or connectSSL when I want and every time I want? Something like this: def check_messages(account): # connect to imap server and check for new mailboxes/messages reactor.connectSSL(...) if __name__ == "__main__": reactor.run() check_message(account1) check_message(account2) ... do something else ... check_message(accountn) Because I can't implement this :} I have started the implementation based on "Twisted Network Programming Essential", but this book doesn't treat multiple connections...
It is possible to: - start the reactor when I initialize my program (eg. in main) - call connectTCP or connectSSL when I want and every time I want?
Something like this:
def check_messages(account): # connect to imap server and check for new mailboxes/messages reactor.connectSSL(...)
if __name__ == "__main__": reactor.run()
check_message(account1) check_message(account2) ... do something else ... check_message(accountn)
Because I can't implement this :}
Something like this (obviously won't work verbatim & may not be the best example, but hopefully will give you the idea): def check_messages(account): # connect to imap server and check for new mailboxes/messages reactor.connectSSL(...) # should return a deferred! def do_something_else(): return defer.succeed(None) # return a deferred here too def something_failed(rslt): print rslt reactor.stop() def call_my_stuff(accountList): # check messages asynchronously checks = [check_message(account) for account in accountList] dl = defer.DeferredList(checks) dl.addCallback(do_something_else) # wait until all messages checked to run dl.addCallback(runner, accountList) # wait till we've done something else dl.addErrback(something_failed) # ALWAYS attach errbacks def runner(accountList): reactor.callLater(0.1, call_my_stuff, accountList) if __name__ == "__main__": accountList = [account1, account2] reactor.callWhenRunning(runner, accountList) reactor.run()
If you're not using twistd to launch your app you'll have to run reactor.run(). Otherwise, the eventloop won't ever start. In terms of exposing something like get_messages()...couldn't that issue the callLater with a very tiny value? Or issue the first connectTCP, which the others chained via addCallback? -J On Fri, Jun 18, 2010 at 11:55 AM, Szabolcs Balogh <bszabolcs@gmail.com> wrote:
On Friday 18 June 2010 20:25:50 Itamar Turner-Trauring wrote:
It sounds good, but if I don't call reactor.run() after connectSSL doesn't happens anything... If I call reactor.run() after connectSSL, it works properly...
There are two types of client connections:
1. Connections you open when you first run the program. Call these before reactor.run() is called.
2. Connections that happen as result of events. Events are all dispatched by the reactor, so e.g. gui click or data received over network connection will result in functions being called by the reactor. These functions can also open new connections.
The key here is to distinguish between the order of the code in the text file, and the order the code runs in. In the following example, the connectTCP will happen *after* reactor.run():
# this will get called 10 seconds after reactor.run(): def happensLater(): reactor.connectTCP(....)
reactor.callLater(10, happensLater) reactor.run()
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
It is possible to: - start the reactor when I initialize my program (eg. in main) - call connectTCP or connectSSL when I want and every time I want?
Something like this:
def check_messages(account): # connect to imap server and check for new mailboxes/messages reactor.connectSSL(...)
if __name__ == "__main__": reactor.run()
check_message(account1) check_message(account2) ... do something else ... check_message(accountn)
Because I can't implement this :}
I have started the implementation based on "Twisted Network Programming Essential", but this book doesn't treat multiple connections...
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Fri, Jun 18, 2010 at 12:55 PM, Szabolcs Balogh <bszabolcs@gmail.com>wrote:
It is possible to: - start the reactor when I initialize my program (eg. in main) - call connectTCP or connectSSL when I want and every time I want?
Yes, this is exactly what you are supposed to do. Call the reactor _once_ and once only. The call connectTCP or whatever whenever is appropriate for your program.
Something like this:
def check_messages(account): # connect to imap server and check for new mailboxes/messages reactor.connectSSL(...)
if __name__ == "__main__": reactor.run()
check_message(account1) check_message(account2) ... do something else ... check_message(accountn)
Because I can't implement this :}
I have started the implementation based on "Twisted Network Programming Essential", but this book doesn't treat multiple connections...
The above won't work, because run() blocks. So in your example here, check_messages() won't be called until after the run() call returns, which means the reactor won't be running, which means check_messages won't work. look into callWhenRunning: http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.interfaces.IR... and do something like: def start_checking(): (do your check_messages stuff here) if __name__ == '__main__': reactor.callWhenRunning(start_checking) reactor.run() using callWhenRunning isn't necessary, but I find that it clarifies things for me sometimes reactor.run() is usually the _last_ thing I call in my Twisted programs, unless I have some shutdown cleanup to do, and even that is usually better done in the reactor shutdown hook (see: http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.interfaces.IR... ) you also need to figure out how to structure your program flow using deferreds, which is not always easy, and depends greatly on your specific application. remember that deferreds are just a way of organizing callback functions you can "chain" deferreds something like this (assuming your functions return Deferreds): d = check_first_one() d.addCallback(check_second_one, ...) etc... which will make check_second_one start after check_first_one completes You can also use a deferredList, which is often handy when you want to keep track of several deferred operations which are operating simultaneously (I haven't used one in a while, so I don't have a simple example off the top of my head, but there's a simple example in the Twisted book) Hope this helps, Kevin Horn
On 18/06/10 16:25, Szabolcs Balogh wrote:
On Friday 18 June 2010 17:49:27 Phil Mayers wrote:
On 18/06/10 15:34, Szabolcs Balogh wrote:
def get_mailbox_list(self): twisted.internet.reactor.run()
The reactor isn't restartable. You can only start it once.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Than how can I start checking for messages for different accounts (When I have finished checking the messages for the first account to start checking for the next account, and so on...)?
Others have already answered this, but one way to explain it: You don't "call the reactor" like a function. You start the reactor, and arrange for *it* to call *your* code in response to events. One type of events being timer ticks.
participants (9)
-
Balogh Szabolcs -
exarkun＠twistedmatrix.com -
Gerrat Rickert -
Glyph Lefkowitz -
Itamar Turner-Trauring -
Jason J. W. Williams -
Kevin Horn -
Phil Mayers -
Szabolcs Balogh