Re: [Twisted-Python] A pseudo-deferred class that can be canceled
Hi Glyph I read through <http://twistedmatrix.com/trac/ticket/990> without trying to get all the details (some are not relevant to me, see below). I'll make a few comments here, then continue in the ticket, supposing there's interest. - It would simplify things to separate discussion of canceling deferreds from discussion of timeouts, which I think you were saying too. The timeout issue seemed to fade in that ticket, which is good. The stuff I posted is just about cancelation, nothing to do with timeouts. - Deferreds are fairly simple, would be good to keep them that way (i.e., not add to the class). Deferreds are very general, you can use them as building blocks to do more complex things, which is what my class does - it's just a deferred in the middle. - Once someone has made a function call, gotten a deferred, added call/errbacks to it, etc., it's gone. It's in flight. Forget about it. Relax and let it happen, etc. It's too complicated to try to stop things from happening that someone else set up to happen in some potentially complex manner. Other code, that the Deferred class itself can't possibly be aware of, may be relying on the deferred firing and at least part of its callback chain being run, etc. The simplest thing to do is to just provide a mechanism whereby the eventual holder of the deferred can opt to trigger their deferred immediately and ignore the final result of the original call (supposing there ever is one). - My class seems a bit like the Operation class proposed by Peaker. But it's simpler because you still only work with normal deferreds. You just get to fire a deferred any time you want to, with any value you want, and to ignore what would have been the result had you not preempively fired it. Those are some reactions to the thread. I'm happy to continue there. Terry
On Tue, Jan 5, 2010 at 6:29 AM, Terry Jones <terry@jon.es> wrote:
- Once someone has made a function call, gotten a deferred, added call/errbacks to it, etc., it's gone. It's in flight. Forget about it.
The thing is, this attitude isn't always reasonable. Deferred is not necessarily the place to implement cancellation, but I think the idiom itself is important. Certainly there are some operations where cancellation is not feasible; perhaps because the underlying APIs do not support cancellation, or because allowing the operation to complete is quicker/easier than trying to interrupt it. But, consider something like the transfer of a 512 TB file; cancellation /is/ possible, through closing the TCP connection, and the operation is significantly expensive, so "just forget it and let it finish" is a somewhat cavalier attitude to take.
complex manner. Other code, that the Deferred class itself can't possibly be aware of, may be relying on the deferred firing and at least part of its callback chain being run, etc. The simplest thing to do is to just provide a mechanism whereby the eventual holder of the deferred can opt to trigger their deferred immediately and ignore the final result of the original call (supposing there ever is one).
I would imagine that you would errback the deferred in the general case of cancellation; if your callback chain doesn't handle errors, well... -- mithrandi, i Ainil en-Balandor, a faer Ambar
On Jan 5, 2010, at 2:12 PM, Tristan Seligmann wrote:
On Tue, Jan 5, 2010 at 6:29 AM, Terry Jones <terry@jon.es> wrote:
- Once someone has made a function call, gotten a deferred, added call/errbacks to it, etc., it's gone. It's in flight. Forget about it.
The thing is, this attitude isn't always reasonable. Deferred is not necessarily the place to implement cancellation, but I think the idiom itself is important. Certainly there are some operations where cancellation is not feasible; perhaps because the underlying APIs do not support cancellation, or because allowing the operation to complete is quicker/easier than trying to interrupt it. But, consider something like the transfer of a 512 TB file; cancellation /is/ possible, through closing the TCP connection, and the operation is significantly expensive, so "just forget it and let it finish" is a somewhat cavalier attitude to take.
And let's not forget, in long-running servers, it's possible to "leak" operations, and the associated resources like file descriptors. When the caller says that they're not interested in the result of an operation any more, the library should clean up as much as possible, otherwise you'll start running out of those resources eventually.
participants (3)
-
Glyph Lefkowitz -
Terry Jones -
Tristan Seligmann