[Twisted-Python] flash's rtmp protocol
RTMP is a secret protocol used by Macromedia flash servers to do fancy interactive/streaming stuff. I found a reference to it on one of Bob Ippolito's TODO lists: http://pythonmac.org/wiki/BobIppolito "TODO: [....] Reverse engineer proprietary RTMP protocol (Flash Communication Server) and implement Python version (Twisted)" I'm also interested in doing this, but am unlikely to start soon. So this is just to let you (Bob) know, lest we do too much in parallel. douglas
On Dec 30, 2003, at 1:52 AM, Douglas Bagnall wrote:
RTMP is a secret protocol used by Macromedia flash servers to do fancy interactive/streaming stuff. I found a reference to it on one of Bob Ippolito's TODO lists:
http://pythonmac.org/wiki/BobIppolito
"TODO: [....] Reverse engineer proprietary RTMP protocol (Flash Communication Server) and implement Python version (Twisted)"
I'm also interested in doing this, but am unlikely to start soon. So this is just to let you (Bob) know, lest we do too much in parallel.
I've given the TCP sessions a cursory glance, and my best guess is that it uses something quite similar to its stateless "Flash Remoting MX" feature. There are implementations of the remoting stuff in PHP and Java, but not Python (last I checked). I have not written any code or documentation yet either way. -bob
Hi, Has anyone written an IRC client in Twisted, using protocol.irc? Do I make a "reactor", or is a reactor only used for making servers? All I need is some really simple code illustrating how to connect to an IRC Server, doing an "identd" to satisfy some IRC Servers, and then having a simple imput line, and an output line... It doesn't have to be fancy, just some really basic example code on how to connect, specify a nick, joining a channel, and typing text into a channel, and of course how to quit. Unfortunately, no such example exists, all the parts seem to be there, but none of the methods in the protocol.irc.IRCClient appear to be implemented, which I suspect I would have to make my own sub-class... but I have no clue how to put all this together. There doesn't seem to be any docs that explain this. John
There's an IRC log bot with the examples.... Look here: http://twistedmatrix.com/documents/examples/ JD wrote:
Hi,
Has anyone written an IRC client in Twisted, using protocol.irc? Do I make a "reactor", or is a reactor only used for making servers? All I need is some really simple code illustrating how to connect to an IRC Server, doing an "identd" to satisfy some IRC Servers, and then having a simple imput line, and an output line... It doesn't have to be fancy, just some really basic example code on how to connect, specify a nick, joining a channel, and typing text into a channel, and of course how to quit.
Unfortunately, no such example exists, all the parts seem to be there, but none of the methods in the protocol.irc.IRCClient appear to be implemented, which I suspect I would have to make my own sub-class... but I have no clue how to put all this together. There doesn't seem to be any docs that explain this.
John
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Tuesday 30 December 2003 08:04 pm, JD wrote:
Hi,
Has anyone written an IRC client in Twisted, using protocol.irc? Do I make a "reactor", or is a reactor only used for making servers? All I need is some really simple code illustrating how to connect to an IRC Server, doing an "identd"
ident is a seperate protocol from IRC. You will need to run a seperate daemon accessible from the Internet to provide this. Twisted doesn't have an ident server. I use the one here http://freshmeat.net/projects/fauxident/
to satisfy some IRC Servers, and then having a simple imput line, and an output line... It doesn't have to be fancy, just some really basic example code on how to connect, specify a nick, joining a channel, and typing text into a channel, and of course how to quit.
Unfortunately, no such example exists, all the parts seem to be there, but none of the methods in the protocol.irc.IRCClient appear to be implemented, which I suspect I would have to make my own sub-class... but I have no clue how to put all this together. There doesn't seem to be any docs that explain this.
John
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Wed, Dec 31, 2003 at 12:11:31PM -0700, Eric Mangold wrote:
On Tuesday 30 December 2003 08:04 pm, JD wrote:
Hi,
Has anyone written an IRC client in Twisted, using protocol.irc? Do I make a "reactor", or is a reactor only used for making servers? All I need is some really simple code illustrating how to connect to an IRC Server, doing an "identd"
ident is a seperate protocol from IRC. You will need to run a seperate daemon accessible from the Internet to provide this. Twisted doesn't have an ident server.
Thanks for pointing out this oversight! Fixed in CVS. Jp
On Dec 31, 2003, at 12:52 PM, Jp Calderone wrote:
ident is a seperate protocol from IRC. You will need to run a seperate daemon accessible from the Internet to provide this. Twisted doesn't have an ident server.
Thanks for pointing out this oversight! Fixed in CVS.
so then, did you just add some example code to the CVS? John
Thanks for pointing out this oversight! Fixed in CVS.
so then, did you just add some example code to the CVS?
John
This means he coded support for ident and added that to cvs (twisted.protocols.ident). Client and server. No he did not code IRC client examples if that's what your asking ? -d
On Dec 31, 2003, at 7:30 PM, darryl wrote:
This means he coded support for ident and added that to cvs (twisted.protocols.ident). Client and server. No he did not code IRC client examples if that's what your asking ?
Why can't anyone post just a teeny little bit of information on how to tie all this together? One of my main problems is where to put my code. Do I put it in the factory class, or do I put it in the Client class? One always points to the other. I suppose it don't really matter, but I'm sure the author(s) of twisted have their OWN ideas of where to put their specific code, but they sure don't seem to have been very sucessful in conveying this info to others. I know it's mostly due to my lack of knowledge, but the only part that's really well documented is the Twisted Web section. Anyway, if anyone has any suggestions or tips on what goes where, please post it here. Inquiring minds want to know, and I'm not the only one who wants to know. John
On Jan 1, 2004, at 2:22 AM, JD wrote:
One of my main problems is where to put my code. Do I put it in the factory class, or do I put it in the Client class? One always points to the other. I suppose it don't really matter, but I'm sure the author(s) of twisted have their OWN ideas of where to put their specific code, but they sure don't seem to have been very sucessful in conveying this info to others.
It depends. The general pattern for Twisted code is to separate protocol logic from application logic. Your protocol subclass should translate from protocol-level events (IRCClient.signedOn, in this case) to application-level events (like a hypothetical 'jd.ChatterRobot.nowOnIRCAs(nick)' method). As for division between "protocol" and "factory" - try to make it so that your factory class can support multiple connected clients at once. It should be small. Any logic associated with a particular connection should go in the protocol class, and any logic associated with all connections simultaneously must at least be referenced from the factory class. Generally it's a bad idea to put that logic actually into the protocol class because that establishes a hard dependency between network code and application logic which makes your application logic harder to write test cases for. This pattern breaks down by necessity when dealing with very network-heavy code, or abstractions over things that have to logically understand IP addresses and the like. However, try to follow the general rule as much as possible even in this kind of code. The reason that nobody has suggested a particular answer to your problem is that the best approach involves keeping your code in an organization that best fits your problem domain, and is not dictated by Twisted. As long as the network events make it from the reactor to your python objects, there are few constraints as to how to organize those objects, and most of those have to do with starting up and shutting down cleanly in the context of bin/twistd.
I know it's mostly due to my lack of knowledge, but the only part that's really well documented is the Twisted Web section.
I know our docs aren't the best, but I think that these cover the relevant organization issues: http://www.twistedmatrix.com/documents/howto/servers http://www.twistedmatrix.com/documents/howto/clients and they have nothing to do with Twisted Web. If you disagree, and there is really a particular documentation problem, open a doc bug in the tracker (http://www.twistedmatrix.com/bugs) and assign it to "hypatia".
Hi, I'm open for suggestions on where I can put my customized code. I seem to have a choice between putting it in the "factory" or in the "protocol", and I cannot seem to decide where the best place would be. My application is simple..... here is all it should do.... 1) establish a connection to an IRC server. 2) send an IRC command, like a "who" command, and extract the info 3) then exit the client I could put my code in the IRCClient's "connectionMade" method. Or I could put it in the factory's "startedConnecting" method. The code would be simple, issue a "who" command, and take it's output and return it. So, in my connectionMade method I would issue the command... then call reactor.stop() which would exit from my reactor.run(). I think the factory's "startedConnecting" might get called first, then somewhere down in this mess, the Client would call me back with the "connectionMade", and I could do it there. I just need to know the appropriate place to put my code. None of these examples in the 'examples' directories make this clear. My next task is to know how to extract the 'who' data back from my initial "who" The ONLY callback I see is the "action()" method, where it has the "msg" as the argument. Is that the one I sub-class to read back the result of the "who"? Which I assume is called back from a deferred. There are a number of other callbacks for the private messages and such. Which one do I use? When testing the code, I'm not getting any callbacks to the "action()" method. Is this because something else is fucked up. I don't suppose anyone on this last has an Mac OS-X system, do they? If so, I would love to establish a VIOP meeting if anyone has the time to help me out. My "iChat" AIM handle is 'jdcrunchman' John
On Wednesday 31 December 2003 04:58 pm, JD wrote:
Hi,
I'm open for suggestions on where I can put my customized code. I seem to have a choice between putting it in the "factory" or in the "protocol", and I cannot seem to decide where the best place would be.
Since this is IRC you shouldn't go issuing commands until your signed on, thus the signedOn method of your protocol would be the right place.
My application is simple..... here is all it should do....
1) establish a connection to an IRC server. 2) send an IRC command, like a "who" command, and extract the info 3) then exit the client
I could put my code in the IRCClient's "connectionMade" method. Or I could put it in the factory's "startedConnecting" method. The code would be simple, issue a "who" command, and take it's output and return it. So, in my connectionMade method I would issue the command... then call reactor.stop() which would exit from my reactor.run().
I think the factory's "startedConnecting" might get called first, then somewhere down in this mess, the Client would call me back with the "connectionMade", and I could do it there. I just need to know the appropriate place to put my code. None of these examples in the 'examples' directories make this clear.
My next task is to know how to extract the 'who' data back from my initial "who"
Yeah, it's not obvious how to do that. I'll send another mail when I've a chance to test something, but New Years is coming up right now... :)
The ONLY callback I see is the "action()" method, where it has the "msg" as the argument. Is that the one I sub-class to read back the result of the "who"?
No. An ACTION in IRC terms is a user doing an /me.
Which I assume is called back from a deferred.
There are a number of other callbacks for the private messages and such. Which one do I use? When testing the code, I'm not getting any callbacks to the "action()" method. Is this because something else is fucked up.
Yeah, your code.
I don't suppose anyone on this last has an Mac OS-X system, do they? If so, I would love to establish a VIOP meeting if anyone has the time to help me out. My "iChat" AIM handle is 'jdcrunchman'
John
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-Eric
On Dec 31, 2003, at 7:43 PM, Eric Mangold wrote:
On Wednesday 31 December 2003 04:58 pm, JD wrote:
Hi,
I'm open for suggestions on where I can put my customized code. I seem to have a choice between putting it in the "factory" or in the "protocol", and I cannot seem to decide where the best place would be.
Since this is IRC you shouldn't go issuing commands until your signed on, thus the signedOn method of your protocol would be the right place.
Ok, good point.... but how do I issue commands? let's say I have a IRCClient class called "testIRC". do I use: testIRC.sendLine('signedOn') to send the command to the IRC server to signon? or do I: testIRC.register('mynick', 'myhost', 'irc.debian.com') then, when I want to do the "who" command, would I do: testIRC.sendLine('who') then, i would expect to call me back, so I would override the "action" method, like so: def action(self, user, channel, msg): < extract the info I want from msg and store it somewhere, probably in an instance variable in testIRC object > reactor.stop() --- to exit the client and disconnect, or would I have to put the reactor.stop() in the factory's "clientConnectionFailed" method, which I would override of course. It's things like this that throw me totally for a loop. Where to put things.... what gets the call back...
Yeah, it's not obvious how to do that. I'll send another mail when I've a chance to test something, but New Years is coming up right now... :)
Yea - and while everyone is partying, i take this wonderful opportunity to get some serious coding done, because the place is so quiet, and I'm too broke to do anything anyway.
No. An ACTION in IRC terms is a user doing an /me.
Ok, if "action" isn't the callback which I override to get the output of the who command, then what is? myInfo? luserClient? luserChannels - Called with the number of channels existant on the server (ALMOST what I want) luserOp - Called with the number of ops logged on to the server (GETTING CLOSER - NOT QUITE) luserMe - Called with information about the server connected to. (CLOSEST YET - but no IP address info) Ok, so how do these get called? As far as I can tell, "handleCommand" calls method = getattr(self, "irc_%s" % command, None) which it then somehow then calls back to: def irc_RPL_LUSERME(self, prefix, params): self.luserMe(params[1]) I really want: RPL_WHOISSERVER, but (sigh) it's not there. Nobody bothered to finish this module. Could I add this to my IRCClient's subclass? IE: def irc_RPL_WHOISSERVER(self, prefix, params): self.whois(params[1]) Then.... def whois(self, prefix, info): < then in this code, get the info? - then after that do the reactor.stop() so I can fall out of the reactor.run() call. > Is this all I have to do.... but then would 'getattr' as you see above, know how to get the method name? Or would I have to add this somehow to another dictionary? If so, how do I do it? and if i have to do it, how do I find the reference to the dictionary? Anyway, other then being totally confused, is there any hope? I think I'm just missing a few critical steps. Or do I have them all?
There are a number of other callbacks for the private messages and such. Which one do I use? When testing the code, I'm not getting any callbacks to the "action()" method. Is this because something else is fucked up.
Yeah, your code.
No shit sherlock.... now i got to figure out why it's not calling my "whois" method. John
On Thu, Jan 01, 2004 at 11:25:37AM -0800, JD wrote:
On Dec 31, 2003, at 7:43 PM, Eric Mangold wrote:
Since this is IRC you shouldn't go issuing commands until your signed on, thus the signedOn method of your protocol would be the right place.
Ok, good point.... but how do I issue commands? let's say I have a IRCClient class called "testIRC".
do I use: testIRC.sendLine('signedOn') to send the command to the IRC server to signon?
No. IRCClient is a LineReceiver subclass, because that's how the low-level protocol works. However, you're not using the low-level protocol, you're using an abstraction on top of it -- IRCClient. Don't break the abstraction (by calling LineReceiver's sendLine method) unless you have a good reason to and you know the protocol spec well enough to know what you're doing (which would be RFC 2812 in this case, I think). As far as I'm aware, sending a line saying 'signedOn' to the server doesn't make any sense in the IRC protocol, so this definitely wouldn't achieve anything useful, except causing an error from the server that the IRCClient instance probably won't expect (because you broke the abstraction and snuck behind its back to do it).
or do I: testIRC.register('mynick', 'myhost', 'irc.debian.com')
That looks much saner. And noticeably more full of information than testIRC.sendLine('signedOn')...
then, when I want to do the "who" command, would I do: testIRC.sendLine('who')
Well, I believe the WHO command requires an argument, so it would be more like testIRC.sendLine('WHO ' + name)... but see my next answer.
then, i would expect to call me back, so I would override the "action" method, like so:
def action(self, user, channel, msg): < extract the info I want from msg and store it somewhere, probably in an instance variable in testIRC object > reactor.stop() --- to exit the client and disconnect, or would I have to put the reactor.stop() in the factory's "clientConnectionFailed" method, which I would override of course.
That's not what the action method is for. As the docstring for action says: "Called when I see a user perform an ACTION on a channel." An ACTION is a specific thing in IRC, not a catch-all for every type of message from server to client. See the RFC if you're not familar with the terms. The right way to do this is implemented in sandbox/exarkun/irc2.py -- see the AdvancedClient.who method. I'd follow JP's earlier suggestion and use it.
It's things like this that throw me totally for a loop. Where to put things.... what gets the call back...
It depends on the protocol implementation. There's no single generic answer for the whole of Twisted.
No. An ACTION in IRC terms is a user doing an /me.
Ok, if "action" isn't the callback which I override to get the output of the who command, then what is?
myInfo? luserClient?
luserChannels - Called with the number of channels existant on the server (ALMOST what I want) luserOp - Called with the number of ops logged on to the server (GETTING CLOSER - NOT QUITE) luserMe - Called with information about the server connected to. (CLOSEST YET - but no IP address info)
The answer is of course none of those. You should've already noticed that IRCClient doesn't implement a who() method, so it's hardly surprising it hasn't implemented a command to deal with its response. irc2.py does, though (and uses Deferreds for dealing with the response, which is cleaner anyway).
Ok, so how do these get called? As far as I can tell, "handleCommand" calls
method = getattr(self, "irc_%s" % command, None)
which it then somehow then calls back to:
def irc_RPL_LUSERME(self, prefix, params): self.luserMe(params[1])
I really want: RPL_WHOISSERVER, but (sigh) it's not there. Nobody bothered to finish this module.
I thought you wanted to use the WHO command not WHOIS? Please try to be precise. Anyway, irc2.py also implements WHOIS.
Could I add this to my IRCClient's subclass?
IE:
def irc_RPL_WHOISSERVER(self, prefix, params): self.whois(params[1])
If you look at irc2.py, you'll see it does override this method. Note that 'whois' is a bad name for the response handler -- the existing convention in IRCClient is that 'join', 'nick', etc, are the methods that *issue* the command, not that handle the response.
Is this all I have to do.... but then would 'getattr' as you see above, know how to get the method name? Or would I have to add this somehow to another dictionary?
Yes, it would. Please see the docs for getattr in the Python standard library reference.
Anyway, other then being totally confused, is there any hope? I think I'm just missing a few critical steps. Or do I have them all?
No idea. You tell me :) I think the biggest critical step you're missing is that you're busily trying to reinvent a wheel that's already been made in sandbox/exarkun/irc2.py, though. -Andrew.
On Jan 1, 2004, at 6:52 PM, Andrew Bennetts wrote:
On Thu, Jan 01, 2004 at 11:25:37AM -0800, JD wrote:
On Dec 31, 2003, at 7:43 PM, Eric Mangold wrote:
Since this is IRC you shouldn't go issuing commands until your signed on, thus the signedOn method of your protocol would be the right place.
Ok, good point.... but how do I issue commands? let's say I have a IRCClient class called "testIRC".
do I use: testIRC.sendLine('signedOn') to send the command to the IRC server to signon?
No. IRCClient is a LineReceiver subclass, because that's how the low-level protocol works.
I think I understand. what is needed in the docs, is a better description of just what the roles of these objects are supposed to do, and where we, as users of Twisted, would have a much better chance to use Twisted in the way it is intended. So the usage of 'sendLine' in a 'lineReceiver' would be inappropriate? but if that's true, then how does one send a command to the IRC server? By the way, I've chosen this as a simple test application, not unlike what I want to really do, so I have chosen a relatively simple application, where I just extract all the users of a specified IRC server, and return the data. I have not specified the form of the data, that's not important, but what's important to me, is the understand how to use Twisted in the way it was intended. I have this: class IRC_test(irc.IRCClient): def whois_on(self): """ This method issues a 'WHO' command to see who is on the server. """ self.sendLine("WHO") def signedOn(self): """Called after sucessfully signing on to the server. So we immediately issue the 'whois_on' message to let the server go and do it's stuff. """ self.whois_on(self) def whois_result(self, prefix, info): """ This is the code the IRC is somehow going to call. Then, after it is called, it stores the info in the factory and returns by calling 'reactor.stop()' to exit the reactor.run() """ self.result = extract_whois_data(info) reactor.stop() def irc_RPL_WHOISSERVER(self, prefix, params): """ This was not included in the parent class IRCClient, So I add it here. 'lineReceived' gets the command = numeric_to_symbolic[command], which is now the string 'RPL_WHOISSERVER'. To get THIS method to call, the 'handleCommand' method first generates a key to the class's 'attribute' dictionary, by pre-pending 'irc_' in front of it to get this method, which is then called. 'params' would then have the result data. We need to hack around to see what form it's in. """ self.whois_result(params[1]) def test_irc(host, port): """ This function scans a particular IRC server, and returns a dictionary of all the nicks and IP addresses. Nick is the key, the IP or reverse dns is the value. """ my_nick = "jd6969" data_dict = {} # place where I want my IRC data myTestFactory = IRC_test(data_dict, my_nick) reactor.connectTCP(host, port, myTestFactory) reactor.run() # I just wait for the server to respond return data_dict # when it does, I just want to return with the data I know this is a pretty stupid and useless application, but this is very close to what I want it to do, without revealing anything. I haven't tested this yet, because I just want to be sure I'm doing it right, rather then to spend weeks down a particular path to nowhere.
However, you're not using the low-level protocol, you're using an abstraction on top of it -- IRCClient. Don't break the abstraction (by calling LineReceiver's sendLine method) unless you have a good reason to and you know the protocol spec well enough to know what you're doing (which would be RFC 2812 in this case, I think).
I believe I do, because I still have to tell the IRC server what I want it to do. And the only way I can see, is with the sendLine command.
As far as I'm aware, sending a line saying 'signedOn' to the server doesn't make any sense in the IRC protocol, so this definitely wouldn't achieve anything useful, except causing an error from the server that the IRCClient instance probably won't expect (because you broke the abstraction and snuck behind its back to do it).
Ok, so it is clear I'm doing this wrong. If that's the case, then what is the RIGHT way of doing it?
or do I: testIRC.register('mynick', 'myhost', 'irc.debian.com')
That looks much saner. And noticeably more full of information than testIRC.sendLine('signedOn')...
then, when I want to do the "who" command, would I do: testIRC.sendLine('who')
Well, I believe the WHO command requires an argument, so it would be more like testIRC.sendLine('WHO ' + name)... but see my next answer.
hmmm - in checking the RFC it mentions I can either "WHO <channel>" or "WHO" without specifying a channel, in which it gets the names of ALL the users of the IRC server not hidden.
then, i would expect to call me back, so I would override the "action" method, like so:
def action(self, user, channel, msg): < extract the info I want from msg and store it somewhere, probably in an instance variable in testIRC object > reactor.stop() --- to exit the client and disconnect, or would I have to put the reactor.stop() in the factory's "clientConnectionFailed" method, which I would override of course.
That's not what the action method is for. As the docstring for action says: "Called when I see a user perform an ACTION on a channel." An ACTION is a specific thing in IRC, not a catch-all for every type of message from server to client. See the RFC if you're not familar with the terms.
Yes - I was mislead... I interpreted this as an action that might come from the server, such as the result of a WHO command (in using my example) I think if you take a look at the shell of my application, I'm no longer using this.
The right way to do this is implemented in sandbox/exarkun/irc2.py -- see the AdvancedClient.who method. I'd follow JP's earlier suggestion and use it.
I was NOT aware that anything in "sandbox" had anything to do with IRC. Expecially when words like this are chosen. One can interpret 'sandbox' in many zillion ways, so how would I even know this. I took a look at this, and WOW - this might be just what I need. I'm going to have to spend a lot of time studying this. I had no clue this even existed, primarily because the name 'sandbox' really had nothing to do with IRC, and I just couldn't make the connection.
The answer is of course none of those. You should've already noticed that IRCClient doesn't implement a who() method, so it's hardly surprising it hasn't implemented a command to deal with its response.
yes - as it also doesn't implement a LOT of things, and yet they are defined as overrideable functions.
irc2.py does, though (and uses Deferreds for dealing with the response, which is cleaner anyway).
Kewl - and now just being aware of that, I definately want to check it out. And also I could use some connents on my above approach, as I have avoided the pitfalls you outlined above.
I thought you wanted to use the WHO command not WHOIS? Please try to be precise.
I thought they were almost the same.... at least that's what I read in the RFC's. What is the difference?
Anyway, irc2.py also implements WHOIS.
Kewl
Could I add this to my IRCClient's subclass?
IE:
def irc_RPL_WHOISSERVER(self, prefix, params): self.whois(params[1])
If you look at irc2.py, you'll see it does override this method. Note that 'whois' is a bad name for the response handler -- the existing convention in IRCClient is that 'join', 'nick', etc, are the methods that *issue* the command, not that handle the response.
I haven't seen any mention of naming conventions, but if there is one, I certainly would want to know where it is.
I think the biggest critical step you're missing is that you're busily trying to reinvent a wheel that's already been made in sandbox/exarkun/irc2.py, though.
that is obviously true.... I explained above why I just didn't even know of it's existance until it was pointed out to me. John
On Fri, Jan 02, 2004 at 06:13:39PM -0800, JD wrote:
On Jan 1, 2004, at 6:52 PM, Andrew Bennetts wrote:
On Thu, Jan 01, 2004 at 11:25:37AM -0800, JD wrote:
do I use: testIRC.sendLine('signedOn') to send the command to the IRC server to signon?
No. IRCClient is a LineReceiver subclass, because that's how the low-level protocol works.
I think I understand. what is needed in the docs, is a better description of just what the roles of these objects are supposed to do, and where we, as users of Twisted, would have a much better chance to use Twisted in the way it is intended. So the usage of 'sendLine' in a 'lineReceiver' would be inappropriate? but if that's true, then how does one send a command to the IRC server?
No, using sendLine in a LineReceiver is fine -- that's why LineReceiver defines sendLine. What isn't so good is using sendLine in an IRCClient subclass. While IRCClient subclasses LineReceiver, in general you should be using the abstractions IRCClient provides to send commands to and receive responses from the server (if you don't want to use them, then why not just use a plain LineReceiver?). The only reason you should have for calling sendLine in an IRCClient subclass is to extend its functionality to support more of the protocol than it already does. IRCClient already has support for signing on (the 'register' method and the 'signedOn' handler), so you shouldn't use sendLine for that. IRCClient currently lacks some IRC features, such as WHO, so to do those you do need sendLine. Also, good style suggests that you probably shouldn't mix your application code with your IRCClient extensions, i.e. subclass IRCClient for WHO support (and other other unimplemented commands you need), and then subclass that for your application.
By the way, I've chosen this as a simple test application, not unlike what I want to really do, so I have chosen a relatively simple application, where I just extract all the users of a specified IRC server, and return the data.
I have not specified the form of the data, that's not important, but what's important to me, is the understand how to use Twisted in the way it was intended.
I have this:
class IRC_test(irc.IRCClient):
def whois_on(self): """ This method issues a 'WHO' command to see who is on the server. """ self.sendLine("WHO")
def signedOn(self): """Called after sucessfully signing on to the server. So we immediately issue the 'whois_on' message to let the server go and do it's stuff. """ self.whois_on(self)
def whois_result(self, prefix, info): """ This is the code the IRC is somehow going to call. Then, after it is called, it stores the info in the factory and returns by calling 'reactor.stop()' to exit the reactor.run() """ self.result = extract_whois_data(info) reactor.stop()
(This identation is really painful.)
def irc_RPL_WHOISSERVER(self, prefix, params): """ This was not included in the parent class IRCClient, So I add it here. 'lineReceived' gets the command = numeric_to_symbolic[command], which is now the string 'RPL_WHOISSERVER'. To get THIS method to call, the 'handleCommand' method first generates a key to the class's 'attribute' dictionary, by pre-pending 'irc_' in front of it to get this method, which is then called. 'params' would then have the result data. We need to hack around to see what form it's in. """ self.whois_result(params[1])
(This identation won't even work!) But creating a method called 'irc_RPL_WHOISSERVER' will do what you want here. Handler methods with a common prefix that are dispatched with a getattr call are a common, and very nice, idiom in Twisted.
def test_irc(host, port): """ This function scans a particular IRC server, and returns a dictionary of all the nicks and IP addresses. Nick is the key, the IP or reverse dns is the value. """ my_nick = "jd6969" data_dict = {} # place where I want my IRC data myTestFactory = IRC_test(data_dict, my_nick) reactor.connectTCP(host, port, myTestFactory) reactor.run() # I just wait for the server to respond return data_dict # when it does, I just want to return with the data
(Ugh, and now you're using tabs...) IRC_test is a protocol, but you're passing it as a factory to connectTCP. See http://twistedmatrix.com/documents/howto/clients.
I know this is a pretty stupid and useless application, but this is very close to what I want it to do, without revealing anything. I haven't tested this yet, because I just want to be sure I'm doing it right, rather then to spend weeks down a particular path to nowhere.
This code is short. Why didn't you run it yourself to get some idea of how close it is to working? Working code is usually a good sign that you're close to the right path, if not already on it...
However, you're not using the low-level protocol, you're using an abstraction on top of it -- IRCClient. Don't break the abstraction (by calling LineReceiver's sendLine method) unless you have a good reason to and you know the protocol spec well enough to know what you're doing (which would be RFC 2812 in this case, I think).
I believe I do, because I still have to tell the IRC server what I want it to do. And the only way I can see, is with the sendLine command.
Only because you need to extend it to cover parts of the protocol that aren't implemented. You started off talking about 'signedOn', which is already covered by IRCClient, and only later mentioned trying to do a WHO command, which isn't. Confusing issues like this makes it hard to figure out what you're trying to do.
As far as I'm aware, sending a line saying 'signedOn' to the server doesn't make any sense in the IRC protocol, so this definitely wouldn't achieve anything useful, except causing an error from the server that the IRCClient instance probably won't expect (because you broke the abstraction and snuck behind its back to do it).
Ok, so it is clear I'm doing this wrong. If that's the case, then what is the RIGHT way of doing it?
Calling register, of course. Where in the RFC does it say 'signedOn' is a recognised command?
or do I: testIRC.register('mynick', 'myhost', 'irc.debian.com')
That looks much saner. And noticeably more full of information than testIRC.sendLine('signedOn')...
then, when I want to do the "who" command, would I do: testIRC.sendLine('who')
Well, I believe the WHO command requires an argument, so it would be more like testIRC.sendLine('WHO ' + name)... but see my next answer.
hmmm - in checking the RFC it mentions I can either "WHO <channel>" or "WHO" without specifying a channel, in which it gets the names of ALL the users of the IRC server not hidden.
Fair enough. Believe it or not, but I actually know very very little about IRC, I mainly figured it out for that email to you by skimming irc.py and one or two brief google searches to confirm e.g. that 'signedOn' doesn't appear in the RFC.
The right way to do this is implemented in sandbox/exarkun/irc2.py -- see the AdvancedClient.who method. I'd follow JP's earlier suggestion and use it.
I was NOT aware that anything in "sandbox" had anything to do with IRC. Expecially when words like this are chosen. One can interpret 'sandbox' in many zillion ways, so how would I even know this.
Because of a mail from earlier in the thread from JP: http://twistedmatrix.com/pipermail/twisted-python/2004-January/006893.html Which part of "WHO is one of the commands I did manage to improve, but I never moved any of the work out of my sandbox. You can find it in Twisted/sandbox/exarkun/irc2.py." was unclear?
I took a look at this, and WOW - this might be just what I need. I'm going to have to spend a lot of time studying this. I had no clue this even existed, primarily because the name 'sandbox' really had nothing to do with IRC, and I just couldn't make the connection.
Hence the email.
The answer is of course none of those. You should've already noticed that IRCClient doesn't implement a who() method, so it's hardly surprising it hasn't implemented a command to deal with its response.
yes - as it also doesn't implement a LOT of things, and yet they are defined as overrideable functions.
I don't understand what you're saying here. You mean things like irc_RPL_WHOISSERVER? That's just the way the prefixed getattr idiom works; all you need to do support a new command define a handler method for it, and that's it. No explicit registration or anything like that needed. Python's a dynamic language, and Twisted uses that to its advantage.
I thought you wanted to use the WHO command not WHOIS? Please try to be precise.
I thought they were almost the same.... at least that's what I read in the RFC's. What is the difference?
No idea. Again, I don't know IRC very well :) I notice that irc2.py implements both WHO and WHOIS seperately, though, so I figure there are some differences, which is why you saying WHO and then later WHOIS confused me.
Could I add this to my IRCClient's subclass?
IE:
def irc_RPL_WHOISSERVER(self, prefix, params): self.whois(params[1])
If you look at irc2.py, you'll see it does override this method. Note that 'whois' is a bad name for the response handler -- the existing convention in IRCClient is that 'join', 'nick', etc, are the methods that *issue* the command, not that handle the response.
I haven't seen any mention of naming conventions, but if there is one, I certainly would want to know where it is.
It's not explicitly written anywhere that I know of. It was just obvious to me from skimming the code of irc.py and irc2.py that that's what the convention was, because that's what the existing code *does*. I generally try to follow the conventions of code I'm using to avoid getting horribly confused.
I think the biggest critical step you're missing is that you're busily trying to reinvent a wheel that's already been made in sandbox/exarkun/irc2.py, though.
that is obviously true.... I explained above why I just didn't even know of it's existance until it was pointed out to me.
Yes, but it was pointed out to you before this... -Andrew.
On Sat, Jan 03, 2004 at 02:20:31PM +1100, I wrote:
I don't understand what you're saying here. You mean things like irc_RPL_WHOISSERVER? That's just the way the prefixed getattr idiom works; all you need to do support a new command define a handler method for it, and
^ obviously I meant to insert the word "is" here. -Andrew.
On Jan 3, 2004, at 8:32 AM, Andrew Bennetts wrote:
On Sat, Jan 03, 2004 at 02:20:31PM +1100, I wrote:
I don't understand what you're saying here. You mean things like irc_RPL_WHOISSERVER?
Yes, exactly.
That's just the way the prefixed getattr idiom works; all you need to do is support a new command then define a handler method for it
I think I did that, but not sure I did it right. I sent a code fragment up to the list earlier, did you see how I did it? Was I on the right track? John
On Jan 2, 2004, at 7:20 PM, Andrew Bennetts wrote:
The only reason you should have for calling sendLine in an IRCClient subclass is to extend its functionality to support more of the protocol than it already does. IRCClient already has support for signing on (the 'register' method and the 'signedOn' handler), so you shouldn't use sendLine for that. IRCClient currently lacks some IRC features, such as WHO, so to do those you do need sendLine.
That's what I thought. Did you take a look at that code sample I sent earlier? I hacked that up myself, with few examples, so I can see where i might have put things in places they weren't intended to be. I want to really learn Twisted, and to be able to put pieces together without having to be constantly digging for example code, and twiddling with that.
Also, good style suggests that you probably shouldn't mix your application code with your IRCClient extensions, i.e. subclass IRCClient for WHO support (and other other unimplemented commands you need), and then subclass that for your application.
I don't quite understand. Because in my case, my application just intantiates a client long enough to extract some information from the IRC server, issue a "who" command and extract the data. I always thought it was appropriate to sub-class the part i want to do, or parts that are not implemented by the parent, as in my case of wanting to do the "who" command, so I treated the command like any other IRC command that issues commands and gets back data. So If I can't mix my application code with a sub_class of a IRCClient, then how is the right way to do it. John
On Mon, Jan 05, 2004 at 06:59:26PM -0800, JD wrote:
That's what I thought. Did you take a look at that code sample I sent earlier? I hacked that up myself, with few examples, so I can see where i might have put things in places they weren't intended to be.
I did. The basic idea seemed fine, except for the trivial flaws I pointed out. I don't have time to test and debug your code for you, though, especially when it's obvious you haven't tried to do so either. If your code isn't working, then it would help us if you could specify *how* it isn't working, i.e. what error or misbehaviour you are seeing. Your previous mail didn't indicate that you'd tested the example code you sent (and the trivial syntax errors suggest this is also the case).
Also, good style suggests that you probably shouldn't mix your application code with your IRCClient extensions, i.e. subclass IRCClient for WHO support (and other other unimplemented commands you need), and then subclass that for your application.
I don't quite understand. Because in my case, my application just intantiates a client long enough to extract some information from the IRC server, issue a "who" command and extract the data.
I always thought it was appropriate to sub-class the part i want to do, or parts that are not implemented by the parent, as in my case of wanting to do the "who" command, so I treated the command like any other IRC command that issues commands and gets back data.
So If I can't mix my application code with a sub_class of a IRCClient, then how is the right way to do it.
What I was trying to say is rather than subclass like this: IRCClient (from twisted.protocols.irc) | \--YourIRCApp (adds WHO command, has your application logic) It might be a better idea to do it like this: IRCClient (from twisted.protocols.irc) | \--ExtendedIRCClient (adds WHO command) | \--YourIRCApp (has your application logic) But that's really just a cosmetic issue. In an ideal world, Twisted would already have the WHO command implemented in its standard IRCClient. Until we reach that point, though, I still like the idea of keeping the protocol implementation and the application using that protocol seperate, even if the "protocol implementation" part of that picture is currently split between what Twisted already provides (IRCClient) and what you need to add (what I called ExtendedIRCClient above). Or you could not implement WHO yourself at all, and use the more complete IRC implementation in irc2.py... -Andrew.
On Jan 5, 2004, at 7:28 PM, Andrew Bennetts wrote:
On Mon, Jan 05, 2004 at 06:59:26PM -0800, JD wrote:
That's what I thought. Did you take a look at that code sample I sent earlier? I hacked that up myself, with few examples, so I can see where i might have put things in places they weren't intended to be.
I did. The basic idea seemed fine, except for the trivial flaws I pointed out. I don't have time to test and debug your code for you, though, especially when it's obvious you haven't tried to do so either. If your code isn't working, then it would help us if you could specify *how* it isn't working, i.e. what error or misbehaviour you are seeing. Your previous mail didn't indicate that you'd tested the example code you sent (and the trivial syntax errors suggest this is also the case).
Don't worry about the code.... it was put up there to only show to general construct, and how I plan to put things together. it was a simple lame example totally different then what my intended APP was going to be, I just used that just to illustrate to others, my understanding, or lack thereof..
Also, good style suggests that you probably shouldn't mix your application code with your IRCClient extensions, i.e. subclass IRCClient for WHO support (and other other unimplemented commands you need), and then subclass that for your application.
I don't quite understand. Because in my case, my application just intantiates a client long enough to extract some information from the IRC server, issue a "who" command and extract the data.
I always thought it was appropriate to sub-class the part i want to do, or parts that are not implemented by the parent, as in my case of wanting to do the "who" command, so I treated the command like any other IRC command that issues commands and gets back data.
So If I can't mix my application code with a sub_class of a IRCClient, then how is the right way to do it.
What I was trying to say is rather than subclass like this:
IRCClient (from twisted.protocols.irc) | \--YourIRCApp (adds WHO command, has your application logic)
It might be a better idea to do it like this:
IRCClient (from twisted.protocols.irc) | \--ExtendedIRCClient (adds WHO command) | \--YourIRCApp (has your application logic)
I've already decided that.... because when I first started to use Twisted, I had no clue of even the existance of ExtendedIRCClient because it was so cleverly mis-labeled and hidden so well in the examples, I never would have found it, had I not posted this in the first place. John
On Thu, Jan 08, 2004 at 01:04:18PM -0800, JD wrote:
On Jan 5, 2004, at 7:28 PM, Andrew Bennetts wrote:
On Mon, Jan 05, 2004 at 06:59:26PM -0800, JD wrote: [...]
So If I can't mix my application code with a sub_class of a IRCClient, then how is the right way to do it.
What I was trying to say is rather than subclass like this:
IRCClient (from twisted.protocols.irc) | \--YourIRCApp (adds WHO command, has your application logic)
It might be a better idea to do it like this:
IRCClient (from twisted.protocols.irc) | \--ExtendedIRCClient (adds WHO command) | \--YourIRCApp (has your application logic)
I've already decided that.... because when I first started to use Twisted, I had no clue of even the existance of ExtendedIRCClient because it was so cleverly mis-labeled and hidden so well in the examples, I never would have found it, had I not posted this in the first place.
You misunderstand me. I wasn't referring to the AdvancedClient class in sandbox/exarkun/irc2.py, as you seem to be thinking. I was offering a suggestion on how to structure *your* code that extends twisted.protocols.irc.IRCClient, if you choose to extend it. (The fact that AdvancedClient is an extension to IRCClient that subclasses it as I describe above suggests that my advice is sound, but the existence of AdvancedClient is incidental to the point I was trying to make.) -Andrew.
On Wed, Dec 31, 2003 at 03:58:50PM -0800, JD wrote:
Hi,
I'm open for suggestions on where I can put my customized code. I seem to have a choice between putting it in the "factory" or in the "protocol", and I cannot seem to decide where the best place would be.
My application is simple..... here is all it should do....
1) establish a connection to an IRC server. 2) send an IRC command, like a "who" command, and extract the info 3) then exit the client
I could put my code in the IRCClient's "connectionMade" method. Or I could put it in the factory's "startedConnecting" method. The code would be simple, issue a "who" command, and take it's output and return it. So, in my connectionMade method I would issue the command... then call reactor.stop() which would exit from my reactor.run().
startedConnecting isn't the right place. connectionMade is a little closer, but since you need to log in (tell the IRC server your nickname, etc) before you can issue commands like WHO, it isn't totally appropriate either. signedOn is the best place, since it is called by IRCClient once the server acknowledges the login information.
I think the factory's "startedConnecting" might get called first, then somewhere down in this mess, the Client would call me back with the "connectionMade", and I could do it there. I just need to know the appropriate place to put my code. None of these examples in the 'examples' directories make this clear.
My next task is to know how to extract the 'who' data back from my initial "who"
Indeed. This is not very easy to do with IRCClient, since the response the server sends doesn't have a whole lot to associate it with the WHO command you originally issued. You need to keep track of outstanding commands and try to match them up with server replies. I recognized this deficiency some time ago and did some initial work towards improving the API. WHO is one of the commands I did manage to improve, but I never moved any of the work out of my sandbox. You can find it in Twisted/sandbox/exarkun/irc2.py. It is under the same license as the rest of Twisted, even though it currently lacks any indication fo that. If it would be helpful to you, please feel free to use it. The improvements take one general form: commands which will produce server responses are exposed as functions which return a Deferred. When the server replies, the parsed information will be passed to that Deferred's callback. Here is a simple example of how you might use it (untested): class WhoingIRCClient(irc2.AdvancedClient): def signedOn(self): # Find everyone IRC'ing from Twisted Matrix's server. self.who('*twistedmatrix.com' ).addCallbacks(self._cbWho, self._ebWho ).addBoth(self._shutdownEverything ) def _cbWho(self, results): print 'channel user host server nick flags hops realname' for r in results: print ', '.join(r) def _ebWho(self, failure) print 'Error in who!' log.err(failure) def _shutdownEverything(self, ignored): reactor.stop() Jp
On Dec 31, 2003, at 11:11 AM, Eric Mangold wrote:
ident is a seperate protocol from IRC. You will need to run a seperate daemon accessible from the Internet to provide this. Twisted doesn't have an ident server.
I use the one here http://freshmeat.net/projects/fauxident/
Thanx, I'll check it out. John
participants (8)
-
Andrew Bennetts -
Bob Ippolito -
darryl -
Douglas Bagnall -
Eric Mangold -
Glyph Lefkowitz -
JD -
Jp Calderone