[Twisted-Python] patch for background processing (delay 0)
I do hereby consigne all copyrights for this patch to Glyph, as required by Twisted: http://www.twistedmatrix.com/developers/contributing Appologies for not using diff, but I am not yet able to download the cygwin package. (perhaps Sunday bandwidth here will be better!) The first part was developed by twisted: def callLater(self, _seconds, _f, *args, **kw): """See twisted.internet.interfaces.IReactorTime.callLater. """ assert callable(_f), "%s is not callable" % _f assert sys.maxint >= _seconds >= 0, \ "%s is not greater than or equal to 0 seconds" % (_seconds,) # patch follows # tple = DelayedCall(seconds() + _seconds, _f, args, kw, if _seconds != 0: ttc = _seconds + seconds() else: ttc = _seconds tple = DelayedCall(ttc, _f, args, kw, # end of patch self._pendingTimedCalls.remove, self._resetCallLater) insort(self._pendingTimedCalls, tple) return tple The second part is my contribution: def runUntilCurrent(self): """Run all pending timed calls. """ if self.threadCallQueue: # Keep track of how many calls we actually make, as we're # making them, in case another call is added to the queue # while we're in this loop. count = 0 for (f, a, kw) in self.threadCallQueue: try: f(*a, **kw) except: log.err() count += 1 del self.threadCallQueue[:count] now = seconds() # patch follows # while self._pendingTimedCalls and (self._pendingTimedCalls[-1].time <= now): # call = self._pendingTimedCalls.pop() do=[] while self._pendingTimedCalls and (self._pendingTimedCalls[-1].time <= now): do.append(self._pendingTimedCalls.pop()) for call in do: # end of patch try: call.called = 1 call.func(*call.args, **call.kw) except: log.deferr() if hasattr(call, "creator"): e = "\n" e += " C: previous exception occurred in " + \ "a DelayedCall created here:\n" e += " C:" e += "".join(call.creator).rstrip().replace("\n","\n C:") e += "\n" log.msg(e) Sincerly, Bill la Forge http://www.geocities.com/laforge49/ Yahoo! India Matrimony: Find your partner online.
On Wed, 2004-05-19 at 00:03, Bill la Forge wrote:
I do hereby consigne all copyrights for this patch to Glyph, as required by Twisted: http://www.twistedmatrix.com/developers/contributing
Appologies for not using diff, but I am not yet able to download the cygwin package. (perhaps Sunday bandwidth here will be better!)
Thanks for sending it on soon anyway, and thanks for reading the contribution guidelines. Good luck with downloading cygwin! I will try to test and apply this over the weekend if nobody else does it sooner.
Please note that this patch was replaced by a "real" one, which included
a suggestion from Jp as well.
Also, the archive dropped part of my email (patch attached...) where I
again consigned the patch to you.
No problem with CygWin, I just had to find the right time slot--mid
morning is generally best here.
Now, if it just passes the regression tests! (Got my fingers crossed!)
Bill
Glyph Lefkowitz
Bill la Forge wrote:
I do hereby consigne all copyrights for this patch to Glyph, as required by Twisted: http://www.twistedmatrix.com/developers/contributing
Appologies for not using diff, but I am not yet able to download the cygwin package. (perhaps Sunday bandwidth here will be better!)
The first part was developed by twisted:
def callLater(self, _seconds, _f, *args, **kw): """See twisted.internet.interfaces.IReactorTime.callLater. """ assert callable(_f), "%s is not callable" % _f assert sys.maxint >= _seconds >= 0, \ "%s is not greater than or equal to 0 seconds" % (_seconds,) # patch follows # tple = DelayedCall(seconds() + _seconds, _f, args, kw, if _seconds != 0: ttc = _seconds + seconds() else: ttc = _seconds tple = DelayedCall(ttc, _f, args, kw, # end of patch self._pendingTimedCalls.remove, self._resetCallLater)
Additionally, I would suggest that _pendingTimedCalls.append(tple) be used instead of the insort() call when _seconds is 0. This speeds up the case of a callLater(0) loop by about 10%.
insort(self._pendingTimedCalls, tple) return tple The second part is my contribution:
def runUntilCurrent(self): """Run all pending timed calls. """ if self.threadCallQueue: # Keep track of how many calls we actually make, as we're # making them, in case another call is added to the queue # while we're in this loop. count = 0 for (f, a, kw) in self.threadCallQueue: try: f(*a, **kw) except: log.err() count += 1 del self.threadCallQueue[:count] now = seconds() # patch follows # while self._pendingTimedCalls and (self._pendingTimedCalls[-1].time <= now): # call = self._pendingTimedCalls.pop()
This approach might be a bit faster than pop in a loop i = bisect.bisect_left(self._pendingTimedCalls, now) do = self._pendingTimedCalls[i:] self._pendingtimedCalls = self._pendingTimedCalls[:i] or it may not be, given how much more memory it allocates.
do=[] while self._pendingTimedCalls and (self._pendingTimedCalls[-1].time <= now): do.append(self._pendingTimedCalls.pop()) for call in do: # end of patch try: call.called = 1 call.func(*call.args, **call.kw) except: log.deferr() if hasattr(call, "creator"): e = "\n" e += " C: previous exception occurred in " + \ "a DelayedCall created here:\n" e += " C:" e += "".join(call.creator).rstrip().replace("\n","\n C:") e += "\n" log.msg(e) Sincerly,
Thanks for the contribution :) By the way, for a slightly less hefty download than cygwin, check out http://unxutils.sourceforge.net/ (kindly pointed out by bear in #twisted). It includes a Windows-native diff tool. Jp
You're quite welcome. I just hope it passes the regression tests!
I did manage, finally, to download cygwin. (Been a few years since I've
used it.) Here in India, bandwidth depends on a lot more than the dialup
connection. ;-) (Though things have improved greatly in the last few
years.)
As for speed, I still suspect that heapq may be faster than insort. I know
there's a few places in the code base where the last element is
inspected (timeout method comes to mind) that would need to be
changed to the first element.
And, of course, Heapq is mising the remove method (I've since coded it).
But then there's the resetter method... might be impacted and I can't
figure it out. ;-(
Bill
Jp Calderone
In my heapq implementation of a similar timer mechanism (outside of Twisted), I did three things that twisted doesn't: - cancelling a timer doesn't try and remove it from the heapq, instead it just marks the timer as dead. - the time when it's supposed to fire is passed to the fire method so that the timer can determine whether it's supposed to actually fire or not (in the case of a time change it is actually added to the queue a second time without being removed so that I don't have to disrupt the heapq). - it's two phase.. I have a list of newly added timers, and a heapq of waiting-to-fire timers. At the end of the iteration I shove the list into the heapq and clear the list. This prevents infinite callLater(0, ...) situations. -bob On May 19, 2004, at 12:53 AM, Bill la Forge wrote:
You're quite welcome. I just hope it passes the regression tests! I did manage, finally, to download cygwin. (Been a few years since I've used it.) Here in India, bandwidth depends on a lot more than the dialup connection. ;-) (Though things have improved greatly in the last few years.) As for speed, I still suspect that heapq may be faster than insort. I know there's a few places in the code base where the last element is inspected (timeout method comes to mind) that would need to be changed to the first element. And, of course, Heapq is mising the remove method (I've since coded it). But then there's the resetter method... might be impacted and I can't figure it out. ;-( Bill
Jp Calderone
wrote: Thanks for the contribution :) By the way, for a slightly less hefty download than cygwin, check out http://unxutils.sourceforge.net/ (kindly pointed out by bear in #twisted). It includes a Windows-native diff tool. Jp
Bill la Forge http://www.geocities.com/laforge49/
Yahoo! India Matrimony: Find your partner online. _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Well, we're in the process of doing your third item, at least--making it two-phase.
Bill
Bob Ippolito
participants (4)
-
Bill la Forge
-
Bob Ippolito
-
Glyph Lefkowitz
-
Jp Calderone