[Twisted-Python] DeferredList and errback
DeferredList is handy, but it isn't quite what I need for FTPClient. Here's my use-case: A file is deemed to be downloaded by FTPClient when both the 226 transfer complete message is received on the control connection, *and* the data connection is closed. DeferredList takes care of this nicely, so long as it works. In the case where the control connection immediately returns an error (e.g. file not found), one errback will be called and the other Deferred will never be called either way, effectively hanging the client. (I've got a testcase for this, but I haven't checked it in yet because having FTP tests passing is a novelty that I don't want to take away from people ;) So ideally I want DeferredList (or something like DeferredList) to wait for all callbacks, as it currently does, but fire its errback as soon as any of its Deferreds fires an errback. Later errbacks and callbacks would be ignored. I realise I'm probably too late to get this into 0.19.0, but I'm wondering what the right way to provide this is: - new class in defer.py (what would it be called?), - change the behaviour of DeferredList to do what I want, or - add a flag to the current DeferredList so it can do either behaviour? -Andrew.
On Sun, 14 Jul 2002 02:55:49 +1000, Andrew Bennetts <andrew-twisted@puzzling.org> wrote:
DeferredList takes care of this nicely, so long as it works.
Rather than adding a whole new class to defer.py, maybe you could make the other deferred's errback get called by the first one?
"DeferOneOrMany", maybe? Tough thing to name. This is probably the right approach, though; consider that there are cases where one error might be all you need, or one success. In those cases you don't want to get a list as your callback, you just want to get the one value.
- change the behaviour of DeferredList to do what I want, or
Definitely not!! There are plenty of cases where you need _all_ the operations to complete regardless of their status of completion.
- add a flag to the current DeferredList so it can do either behaviour?
As I said above, the behavior is really different; the types of the signatures for your callbacks will be different. That says "different class" to me. -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
On Sat, Jul 13, 2002 at 08:18:38PM -0500, Glyph Lefkowitz wrote:
That's what I'm currently attempting to do, and it's messy. You really want to make sure that adding the errback to call the other Deferred's errbacks doesn't interfere in the normal chain of processing for that errback, so you need to muck around with gunk like "lambda e: otherDeferred.errback(e) or e". Worse than that, with 3 or more Deferreds ordering becomes hell... if you have Deferreds A, B and C, then A should call B & C, B should call A & C, and C should call A & B. I don't know of a clean way to arrange that. Plus, it's what I'm currently trying to do, and it doesn't work. Doubtless this is a bug in how I'm doing it, but that's just another argument for having a class to do it for me.
Well, I *definitely* want all callbacks, or just one errback. I can imagine that only one either could be useful too, though. Taking this approach, suddenly we have two new classes to name, both similar and yet and different to DeferredList: all callbacks or one errback, and one callback or one errback, vs. the current all callbacks and all errbacks.
Ok. I wasn't sure if this aspect of its functionality was in use, though I guessed it probably was.
I'm not so sure. Would it be so bad to be able to do this: dl = DeferredList([d1, d2, ...], fireOnOneCallback=1, fireOnOneErrback=1) This way we can get all four variations for free, without having to think of distinct and non-confusing names for them. Of course, "fireOnOneCallback" is a pretty ugly name for a keyword argument. This approach, despite the way it can change the signature of callback and/or errback, seems more general to me. Or is the way you envisage that "DeferOneOrMany" would behave? -Andrew.
On Sun, 14 Jul 2002 14:41:18 +1000, Andrew Bennetts <andrew-twisted@puzzling.org> wrote:
On second thought, you're right -- I like the way that exact signature looks. :-). Go ahead and implement it. -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
On Sun, 14 Jul 2002 02:55:49 +1000, Andrew Bennetts <andrew-twisted@puzzling.org> wrote:
DeferredList takes care of this nicely, so long as it works.
Rather than adding a whole new class to defer.py, maybe you could make the other deferred's errback get called by the first one?
"DeferOneOrMany", maybe? Tough thing to name. This is probably the right approach, though; consider that there are cases where one error might be all you need, or one success. In those cases you don't want to get a list as your callback, you just want to get the one value.
- change the behaviour of DeferredList to do what I want, or
Definitely not!! There are plenty of cases where you need _all_ the operations to complete regardless of their status of completion.
- add a flag to the current DeferredList so it can do either behaviour?
As I said above, the behavior is really different; the types of the signatures for your callbacks will be different. That says "different class" to me. -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
On Sat, Jul 13, 2002 at 08:18:38PM -0500, Glyph Lefkowitz wrote:
That's what I'm currently attempting to do, and it's messy. You really want to make sure that adding the errback to call the other Deferred's errbacks doesn't interfere in the normal chain of processing for that errback, so you need to muck around with gunk like "lambda e: otherDeferred.errback(e) or e". Worse than that, with 3 or more Deferreds ordering becomes hell... if you have Deferreds A, B and C, then A should call B & C, B should call A & C, and C should call A & B. I don't know of a clean way to arrange that. Plus, it's what I'm currently trying to do, and it doesn't work. Doubtless this is a bug in how I'm doing it, but that's just another argument for having a class to do it for me.
Well, I *definitely* want all callbacks, or just one errback. I can imagine that only one either could be useful too, though. Taking this approach, suddenly we have two new classes to name, both similar and yet and different to DeferredList: all callbacks or one errback, and one callback or one errback, vs. the current all callbacks and all errbacks.
Ok. I wasn't sure if this aspect of its functionality was in use, though I guessed it probably was.
I'm not so sure. Would it be so bad to be able to do this: dl = DeferredList([d1, d2, ...], fireOnOneCallback=1, fireOnOneErrback=1) This way we can get all four variations for free, without having to think of distinct and non-confusing names for them. Of course, "fireOnOneCallback" is a pretty ugly name for a keyword argument. This approach, despite the way it can change the signature of callback and/or errback, seems more general to me. Or is the way you envisage that "DeferOneOrMany" would behave? -Andrew.
On Sun, 14 Jul 2002 14:41:18 +1000, Andrew Bennetts <andrew-twisted@puzzling.org> wrote:
On second thought, you're right -- I like the way that exact signature looks. :-). Go ahead and implement it. -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
participants (2)
-
Andrew Bennetts -
Glyph Lefkowitz