integrating inotify into a protocol
hey guys, I'm new to twisted, and I'm trying to figure out how to integrate the inotify support into a client protocol. let's say I have a client protocol which connects to a server and stays connected, like the "Echo" example here: http://twistedmatrix.com/documents/10.2.0/core/howto/clients.html#auto3 I've taken that example a tweaked it so that it echos back to the server, rather than to stdout (see echo.py, attached), and takes its port number as sys.argv[1]. now, I can start the "server": $ cat | socat - tcp4-listen:1234 and start the client: $ python echo.py 1234 Started to connect. Connected. and if I type something into the server, it gets printed on client's stdout, and then echoed back to the server: $ cat | socat - tcp4-listen:1234 blah blah $ python foo.py 1234 Started to connect. Connected. blah life is good. now, let's say I want to integrate inotify support into my persistent client. let's say that whenever a new file appears in /tmp, I want to client to announce this file to the server, over its persistent connection. accordingly, I've added 'def announceNewFile()' to the Echo() protocol class (see echo_inotify, attached). but herein lies the rub: the inotify notifier just takes a list of callback functions, not a list of objects. so how can I get the notifier to call a method on my Echo protocol object? another way to say this is that the protocol/client/factory side of things is all object-oriented, but the inotify side of things is just procedural. n'ere the twain shall meet? specifically, the notifier expects the callback to accept 3 args: _Watch object, path, and mask. so, when you try to pass in a class method as one of its callbacks, you get something like this: exceptions.TypeError: unbound method announceNewFile() must be called with Echo instance as first argument (got _Watch instance instead) halp? -jason
On 03/30/2011 02:21 AM, Jason Pepas wrote:
another way to say this is that the protocol/client/factory side of things is all object-oriented, but the inotify side of things is just procedural. n'ere the twain shall meet?
specifically, the notifier expects the callback to accept 3 args: _Watch object, path, and mask. so, when you try to pass in a class method as one of its callbacks, you get something like this:
exceptions.TypeError: unbound method announceNewFile() must be called with Echo instance as first argument (got _Watch instance instead)
Well, sure. You need to pass in an *instance* method, not a class method i.e. only once your Echo protocol class has been instantiated, do: class Echo(LineReceiver): def connectionMade(self): inotify.whatever(self.announceNewFile, path, mask) But if you want >1 client, you'll probably want something a bit more sophisticated. Specifically, you'll probably want to route the inotify callbacks via an instance method on the Factory, then have the Factory maintain a list of all Protocol instances, and fan out the inotify to all the clients. e.g. class Echo(...): def connectionMade(self): self.factory.clients.append(self) def connectionLost(self, ...): if self in self.factory.clients: self.factory.clients.remove(self) def announceNewFile(self, ...): whatever class Factory(...): protocol = Echo def __init__(self): self.clients = [] def announceNewFile(self, *p, **kw): for client in self.clients: client.announceNewFile(*p, **kw) def main(): f = Factory() inotify.whatever(f.announceNewFile, path, mask) reactor.listenTCP(port, f) reactor.run()
Phil, that's a huge help, thanks so much for pointing me in the right direction. The notion of the factory being persistent and "owning" a protocol for each connection was what I needed to wrap my head around the problem. In fact, when I thought about it some more, I realized that the factory should "own" the inotify watcher. This then makes it straightforward for the factory to pass in its instance method as inotify's callback. I've implemented your suggestions and retooled my "Echo" example as "TmpTell" (attached). When you run it, it creates several persistent clients (one for each port number listed on the command line). Then, each time a file is created in /tmp, each client tells its server about the new file. here's an example usage. first, start up a "server" $ socat - tcp4-listen:1234 and another: $ socat - tcp4-listen:1235 now start up tmptell: $ python tmptell.py 1234 1235 calling TmpTellClientFactory.__init__ calling TmpTellClientFactory.startFactory calling TmpTellClientFactory.startedConnecting calling TmpTellClientFactory.startedConnecting calling TmpTellClientFactory.buildProtocol calling TmpTell.__init__ calling TmpTellClientFactory.buildProtocol calling TmpTell.__init__ finally, in a third terminal, run 'mktemp': $ mktemp /tmp/tmp.1xtuY16NKr you will then see this additional output in the tmptell terminal: calling TmpTellClientFactory.inotifyEventHappened calling TmpTell.announceNewFile calling TmpTell.announceNewFile and your "servers" will look like this: $ socat - tcp4-listen:1234 new file created at FilePath('/tmp/tmp.1xtuY16NKr') $ socat - tcp4-listen:1235 new file created at FilePath('/tmp/tmp.1xtuY16NKr') if you kill one of the servers with CTRL+c, you see this in the tmptell terminal: calling TmpTell.connectionLost calling TmpTellClientFactory.removeProtocolObject calling TmpTellClientFactory.clientConnectionLost and now we run mktemp a second time: $ mktemp /tmp/tmp.L5EeFSdXRJ and now tmptell correctly calls announceNewFile just once, instead of twice: calling TmpTellClientFactory.inotifyEventHappened calling TmpTell.announceNewFile and the server we didn't kill looks like this now: $ socat - tcp4-listen:1235 new file created at FilePath('/tmp/tmp.1xtuY16NKr') new file created at FilePath('/tmp/tmp.L5EeFSdXRJ') Thanks again Phil, I think I've got it now! -jason
participants (2)
-
Jason Pepas -
Phil Mayers