[Twisted-Python] Getting a synchronous interface to a twisted reactor
I'm trying to write a dead-simple interface for an IRC client library, like so: import simpleirc connection = simpleirc.Connect('irc.freenode.net', 6667) channel = connection.join('foo') find_command = re.compile(r'google ([a-z]+)').findall for msg in channel: for t in find_command(msg): channel.say("http://google.com/search?q=%s" % t) Working from the example in the docs<http://twistedmatrix.com/documents/current/core/howto/clients.html#auto5>, I'm running into trouble with the callbacks (the code is a bit lengthy, so I pasted it here <https://gist.github.com/e7e13f074a2691de6371>). The problem is that the call to channel.__next__ needs to be returned when the callback <IRCClient instance>.privmsg is called, there doesn't seem to be a clean option of doing that. I could try to use exceptions or threads, but that seems like the wrong thing here, is there a simpler (blocking?) way of using a twisted reactor that would make this possible? Cheers, Andrey On Wed, Apr 21, 2010 at 10:00 PM, César García <celord@gmail.com> wrote:
+1 Yes, it's great!!
2010/4/21 Kevin Horn <kevin.horn@gmail.com>
This blog series is also totally rock-a-licious.
-- http://celord.blogspot.com/
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Hello Andrey, May I recommend you to look at Twisted Words? There are plenty of IRC tools there. Regards, Alex 2010/4/21 Andrey Fedorov <anfedorov@gmail.com>:
I'm trying to write a dead-simple interface for an IRC client library, like so: import simpleirc connection = simpleirc.Connect('irc.freenode.net', 6667) channel = connection.join('foo') find_command = re.compile(r'google ([a-z]+)').findall for msg in channel: for t in find_command(msg): channel.say("http://google.com/search?q=%s" % t) Working from the example in the docs, I'm running into trouble with the callbacks (the code is a bit lengthy, so I pasted it here). The problem is that the call to channel.__next__ needs to be returned when the callback <IRCClient instance>.privmsg is called, there doesn't seem to be a clean option of doing that. I could try to use exceptions or threads, but that seems like the wrong thing here, is there a simpler (blocking?) way of using a twisted reactor that would make this possible? Cheers, Andrey
On Wed, Apr 21, 2010 at 10:00 PM, César García <celord@gmail.com> wrote:
+1 Yes, it's great!!
2010/4/21 Kevin Horn <kevin.horn@gmail.com>
This blog series is also totally rock-a-licious.
-- http://celord.blogspot.com/
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Alexandre Quessy http://alexandre.quessy.net/
Thanks! The main reason for the question, though, is just curiosity from playing with and learning the Twisted API, not necessarily getting the example working :) A more direct question would have been - is there a Twisted reactor which provides a blocking call instead of a callback? Is there an accepted "best" way of wrapping an non-blocking async API with callbacks into a blocking synchronous one? - Andrey On Thu, Apr 22, 2010 at 12:40 AM, Alexandre Quessy <alexandre@quessy.net>wrote:
Hello Andrey, May I recommend you to look at Twisted Words? There are plenty of IRC tools there.
Regards, Alex
I'm trying to write a dead-simple interface for an IRC client library,
2010/4/21 Andrey Fedorov <anfedorov@gmail.com>: like
so: import simpleirc connection = simpleirc.Connect('irc.freenode.net', 6667) channel = connection.join('foo') find_command = re.compile(r'google ([a-z]+)').findall for msg in channel: for t in find_command(msg): channel.say("http://google.com/search?q=%s" % t) Working from the example in the docs, I'm running into trouble with the callbacks (the code is a bit lengthy, so I pasted it here). The problem is that the call to channel.__next__ needs to be returned when the callback <IRCClient instance>.privmsg is called, there doesn't seem to be a clean option of doing that. I could try to use exceptions or threads, but that seems like the wrong thing here, is there a simpler (blocking?) way of using a twisted reactor that would make this possible? Cheers, Andrey
On Wed, Apr 21, 2010 at 10:00 PM, César García <celord@gmail.com> wrote:
+1 Yes, it's great!!
2010/4/21 Kevin Horn <kevin.horn@gmail.com>
This blog series is also totally rock-a-licious.
-- http://celord.blogspot.com/
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Alexandre Quessy http://alexandre.quessy.net/
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
You can execute blocking call at any time of your pleasing. It will predictable block all execution paths. Typical wrapper around blocking calls is thread. Something like this (from http://twistedmatrix.com/documents/current/core/howto/threading.html): d = threads.deferToThread(yourSyncCall) d.addCallback(...) -- Konrads Smelkovs Applied IT sorcery. On Thu, Apr 22, 2010 at 9:06 AM, Andrey Fedorov <anfedorov@gmail.com> wrote:
Thanks! The main reason for the question, though, is just curiosity from playing with and learning the Twisted API, not necessarily getting the example working :)
A more direct question would have been - is there a Twisted reactor which provides a blocking call instead of a callback? Is there an accepted "best" way of wrapping an non-blocking async API with callbacks into a blocking synchronous one?
- Andrey
On Thu, Apr 22, 2010 at 12:40 AM, Alexandre Quessy <alexandre@quessy.net>wrote:
Hello Andrey, May I recommend you to look at Twisted Words? There are plenty of IRC tools there.
Regards, Alex
I'm trying to write a dead-simple interface for an IRC client library,
2010/4/21 Andrey Fedorov <anfedorov@gmail.com>: like
so: import simpleirc connection = simpleirc.Connect('irc.freenode.net', 6667) channel = connection.join('foo') find_command = re.compile(r'google ([a-z]+)').findall for msg in channel: for t in find_command(msg): channel.say("http://google.com/search?q=%s" % t) Working from the example in the docs, I'm running into trouble with the callbacks (the code is a bit lengthy, so I pasted it here). The problem is that the call to channel.__next__ needs to be returned when the callback <IRCClient instance>.privmsg is called, there doesn't seem to be a clean option of doing that. I could try to use exceptions or threads, but that seems like the wrong thing here, is there a simpler (blocking?) way of using a twisted reactor that would make this possible? Cheers, Andrey
On Wed, Apr 21, 2010 at 10:00 PM, César García <celord@gmail.com> wrote:
+1 Yes, it's great!!
2010/4/21 Kevin Horn <kevin.horn@gmail.com>
This blog series is also totally rock-a-licious.
-- http://celord.blogspot.com/
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Alexandre Quessy http://alexandre.quessy.net/
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On 04/22/2010 07:06 AM, Andrey Fedorov wrote:
Thanks! The main reason for the question, though, is just curiosity from playing with and learning the Twisted API, not necessarily getting the example working :)
A more direct question would have been - is there a Twisted reactor which provides a blocking call instead of a callback? Is there an accepted "best" way of wrapping an non-blocking async API with callbacks into a blocking synchronous one?
If I understand what you're asking you want to call Twisted code in the following style: def function(): # this code blocks result = some_twisted_thing() value = process(result) final = morecode(value) Rather than: def function(): def = some_twisted_thing() def.addCallback(process) def.addCallback(morecode) If that's the case, then the answer is "sort of" but I think you might have misunderstood Twisted a bit. The entire point is to have non-blocking APIs, not to want a blocking API. You can't "really" block, because that stops the reactor from running, and thus any networking events being handled. You can emulate it in Python 2.5 with generators & a special decorator: from twisted.internet import deferred @deferred.inlineCallbacks def function(): result = yield some_twisted_thing() value = yield process(result) final = yield morecode(value) I might have misunderstood what you're asking though, in which case I apologise, and can you be more specific?
On Thu, Apr 22, 2010 at 3:21 AM, Phil Mayers <p.mayers@imperial.ac.uk>wrote:
If that's the case, then the answer is "sort of" but I think you might have misunderstood Twisted a bit. The entire point is to have non-blocking APIs, not to want a blocking API.
That sounds like might be the case case. Wwhat I'm looking for, then, is a way to use the IRCClient protocol outside of Twisted - in a way where I can use it like: connection = # something using IRCClient msg = connection.privmsg() # blocks assert msg == "andreyf: hello there, lovely IRC users!" Are there any projects along those lines? Cheers, Andrey
On 22/04/10 15:50, Andrey Fedorov wrote:
On Thu, Apr 22, 2010 at 3:21 AM, Phil Mayers <p.mayers@imperial.ac.uk <mailto:p.mayers@imperial.ac.uk>> wrote:
If that's the case, then the answer is "sort of" but I think you might have misunderstood Twisted a bit. The entire point is to have non-blocking APIs, not to want a blocking API.
That sounds like might be the case case. Wwhat I'm looking for, then, is a way to use the IRCClient protocol outside of Twisted - in a way where I can use it like:
connection = # something using IRCClient msg = connection.privmsg() # blocks assert msg == "andreyf: hello there, lovely IRC users!"
Are there any projects along those lines?
Not that I'm aware of. I'm pretty certain the Twisted protocol implementations all require the reactor and thus need to be run in an async manner. There are various hacky ways you could accomplish this; the obvious one that springs to mind is running the reactor in a thread. There are various caveats to doing this, the main one being you can't call any Twisted functions outside the reactor thread except "callFromThread" so you'd probably want to write some kind of wrapper; and it's a bit counter to the whole ethos. Given you said: """The main reason for the question, though, is just curiosity from playing with and learning the Twisted API""" ...I would recommend just trying to use the API in the Twisted "way". You can use the aforementioned @inlineCallbacks decorator & generator syntax if you prefer writing your code that way, as opposed to traditional deferred callback chaining.
On Thu, Apr 22, 2010 at 11:00 AM, Phil Mayers <p.mayers@imperial.ac.uk>wrote:
I would recommend just trying to use the API in the Twisted "way". You can use the aforementioned @inlineCallbacks decorator & generator syntax if you prefer writing your code that way, as opposed to traditional deferred callback chaining.
This seems wise :) Thanks for all the help!
participants (4)
-
Alexandre Quessy
-
Andrey Fedorov
-
Konrads Smelkovs
-
Phil Mayers