[Twisted-Python] pb help needed.
I have a bug in my woven application which manifests itself as win32select failing after doing many http requests. The error is the constant WSAENOBUFS and I it seems pretty likely that this is caused by some socket related resource leaking for each request I make. This hypothesis is further supported by observing that a host of pb.ClientFactory instances are stopped when shutting down the application after it crashes. This seems strange to me as I am thorough in shutting down the connector after every request is finished. Turning to that trusty friend printf style debugging, shows that the PBClientFactory is stopped correctly when performing requests to pages that use the referenceable and wait for it's deferred's to complete. There are two cases when it doesn't stop. The first is when I press reload quickly a number of times. Then I just see start messages because my request finish method is not called (it is a callback to the deferred returned by request.notifyFinish()). The second case is when I call disconnect on the connector before it's deferred has been realised. I see in the log the following pattern: Starting pb.PBClientFactory, disconnect and then the attached event, stopping factory is not called at all. After stepping I see that disconnect doesn't do anything when it is called before the connection has been realized. So I have a bug, and it's in my code. My question is how does one manage pb resources correctly and easily (I don't worry about performance just yet) when each user needs to have it's own connection?
Syver Enstad <syver@inout.no> writes:
I have a bug in my woven application which manifests itself as win32select failing after doing many http requests. The error is the constant WSAENOBUFS and I it seems pretty likely that this is caused by some socket related resource leaking for each request I make. This hypothesis is further supported by observing that a host of pb.ClientFactory instances are stopped when shutting down the application after it crashes. This seems strange to me as I am thorough in shutting down the connector after every request is finished.
Turning to that trusty friend printf style debugging, shows that the PBClientFactory is stopped correctly when performing requests to pages that use the referenceable and wait for it's deferred's to complete. There are two cases when it doesn't stop.
The first is when I press reload quickly a number of times. Then I just see start messages because my request finish method is not called (it is a callback to the deferred returned by request.notifyFinish()).
The second case is when I call disconnect on the connector before it's deferred has been realised. I see in the log the following pattern: Starting pb.PBClientFactory, disconnect and then the attached event, stopping factory is not called at all. After stepping I see that disconnect doesn't do anything when it is called before the connection has been realized.
So I have a bug, and it's in my code. My question is how does one manage pb resources correctly and easily (I don't worry about performance just yet) when each user needs to have it's own connection?
Here is what I have come up with so far. It doesn't solve the problem where I don't get the request finish callback, but seems to be able to disconnect even if not connected when disconnect is called. class MyPBClientFactory(pb.PBClientFactory): def disconnect(self): if not self._broker: deferred = self.getRootObject() deferred.addCallback( lambda unused: pb.PBClientFactory.disconnect(self)) else: pb.PBClientFactory.disconnect(self)
Syver Enstad <syver@inout.no> writes:
Syver Enstad <syver@inout.no> writes:
I have a bug in my woven application which manifests itself as win32select failing after doing many http requests. The error is the constant WSAENOBUFS and I it seems pretty likely that this is caused by some socket related resource leaking for each request I make. This hypothesis is further supported by observing that a host of pb.ClientFactory instances are stopped when shutting down the application after it crashes. This seems strange to me as I am thorough in shutting down the connector after every request is finished.
Turning to that trusty friend printf style debugging, shows that the PBClientFactory is stopped correctly when performing requests to pages that use the referenceable and wait for it's deferred's to complete. There are two cases when it doesn't stop.
The first is when I press reload quickly a number of times. Then I just see start messages because my request finish method is not called (it is a callback to the deferred returned by request.notifyFinish()).
The second case is when I call disconnect on the connector before it's deferred has been realised. I see in the log the following pattern: Starting pb.PBClientFactory, disconnect and then the attached event, stopping factory is not called at all. After stepping I see that disconnect doesn't do anything when it is called before the connection has been realized.
So I have a bug, and it's in my code. My question is how does one manage pb resources correctly and easily (I don't worry about performance just yet) when each user needs to have it's own connection?
Here is what I have come up with so far. It doesn't solve the problem where I don't get the request finish callback, but seems to be able to disconnect even if not connected when disconnect is called.
class MyPBClientFactory(pb.PBClientFactory): def disconnect(self): if not self._broker: deferred = self.getRootObject() deferred.addCallback( lambda unused: pb.PBClientFactory.disconnect(self)) else: pb.PBClientFactory.disconnect(self)
Since the last post I moved to a persistent PBClientFactory so that disconnects is not a problem. This fixes the crashes, but there are still resources that are not released. I only see perspective attached messages in the log, until I kill twistd when I get a host of detached messages. My next question must be how do I detach the perspectives in an orderly fashion after each request? This seems to run into the same problems as I had with not being able to do cleanup for each request.
How does one detach a perspective? Is there a callRemote method on a reference that does this or what? Or is it not a problem that a long running server calls getPerspective and unbounded number of times (I have two twisted servers communicating via pb).
On 10 Nov 2003 11:58:18 +0100 Syver Enstad <syver@inout.no> wrote:
How does one detach a perspective? Is there a callRemote method on a reference that does this or what? Or is it not a problem that a long running server calls getPerspective and unbounded number of times (I have two twisted servers communicating via pb).
1.1 no longer has the concept of detaching. You can track live perspectives and choose to not allow users to login if there's still a connection, or not as you will. Look a bit at http://twistedmatrix.com/documents/howto/pclients, and put the appropriate logic in your realm'ds requestAvatar. -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python & Twisted consulting
participants (2)
-
Itamar Shtull-Trauring -
Syver Enstad