[Twisted-Python] Bug in t.w.w.WidgetPage

Hi all,
I think there might be a bug lurking in t.w.w.WidgetPage's handling of Deferreds. Take a look at t.w.w.WidgetPage.callback:
def callback(self, result, position, decNeedsHeaders): if result != FORGET_IT: self.needsHeaders = self.needsHeaders - decNeedsHeaders else: result = [FORGET_IT] for i in xrange(len(result)): if isinstance(result[i], defer.Deferred): self._addDeferred(result[i], position+i) # print 'CALLBACK:',self.lst, position, result if not isinstance(result, types.ListType): result = [result] self.lst[position:position+1] = result assert self.position <= position self.keepRendering() for r in result: if isinstance(r, defer.Deferred): r.arm()
Now consider this sequence of events:
(1) We initialize a t.w.w.WidgetPage with two Deferreds (head and foot), so in our WidgetPage (let us call it foo) we have foo.lst = [head, foot].
(2) mouth is now almost done, so it defers the rest of it's computation, and callback's with a string 'Hello, ' and a Deferred (let us call it mouth).
(3) After foo.callback is done we now have that foo.lst = ['Hello, ', mouth, foot].
(4) Now foot is done and callback's with a string 'World!'.
(5) After foo.callback is done this time we have that foo.lst = ['Hello, ', 'World!', foot]
Whoa! We just put our foot in our mouth! So to speak :-)

[ Sune Kirkeby ]
I think there might be a bug lurking in t.w.w.WidgetPage's handling of Deferreds.
And here is a patch to fix it. The patch does away with Deferred holding their positions in lst, instead a Deferred now hold a sentinel, that is put into lst with the Deferred.
When we need the position in callback we go looking for the sentinel the Deferred gave us. This also allows us to stop keeping data we have written to the request (i.e. lst now only holds the Deferreds and strings that have not been sent).
Also, the patch adds NOT_DONE_YET support to WidgetPage. This works by Deferreds callback'ing with a tuple (NOT_DONE_YET, partial_result) which is insert into lst before the Deferred.
So, now Deferreds can callback to WidgetPage multiple times with partial results. (Not sure if this is abusing the Deferred protocol, but it solves an itch I had :-).
Comments?

On Wed, 2002-01-02 at 16:33, Sune Kirkeby wrote:
[ Sune Kirkeby ]
I think there might be a bug lurking in t.w.w.WidgetPage's handling of Deferreds.
And here is a patch to fix it.
...
Comments?
I haven't had time to review it yet, but thanks a lot! This is a bug that I'd suspected was in twisted.web for some time now, but hadn't had time to verify or test for.
participants (2)
-
Glyph Lefkowitz
-
Sune Kirkeby