[Python-ideas] re-implementing Twisted for fun and profit

[Shane Green]

You make an excellent point about the different levels being discussed.  Yes, you understand my point well.  For some reason I've always hated thinking of the promise as immutable, but that's the normal terminology.  The reality is that a Promise represents the output of an operation, and will emit the output of that operation to all callers that register with it.  The promise doesn't pass itself as the value to the callbacks, so its immutability is somewhat immaterial. I'm not arguing with you on that point, just the general description of the pattern.  The more I think about it, the more I'm realizing how inappropriate something like a deferred or promise is to this discussion.  

Unfortunately my knowledge of coroutines is somewhat limited, and my time the lasts couple of days, and the next couple, is preventing me from giving it a good think through.  I understand them well enough to know they're cool, and I'm pretty sure I like the idea of making them the event loop mechanism.  I think it would be good for us all to continuously revisit concrete examples during the discussion, because the set of core I/O are small enough to revisit multiple times.  If a much more general mechanism naturally falls out then great.  





Shane Green 
www.umbrellacode.com
805-452-9666 | shane at umbrellacode.com

On Oct 15, 2012, at 8:51 AM, Glyph <glyph at twistedmatrix.com> wrote:

> 
> On Oct 15, 2012, at 1:03 AM, Shane Green <shane at umbrellacode.com> wrote:
> 
>> Namely, all callbacks registered with a given Promise instance, receive the output of the original operation
> 
> This is somewhat tangential to the I/O loop discussion, and my hope for that discussion is that it won't involve Deferreds, or Futures, or Promises, or any other request/response callback management abstraction, because requests and responses are significantly higher level than accept() and recv() and do not belong within the same layer.  The event loop ought to provide tools to experiment with event-driven abstractions so that users can use Deferreds and Promises - which are, fundamentally, perfectly interoperable, and still use standard library network protocol implementations.
> 
> What I think you were trying to say was that callback addition on Deferreds is a destructive operation; whereas your promises are (from the caller's perspective, at least) immutable.  Sometimes I do think that the visibly mutable nature of Deferreds was a mistake.  If I read you properly though, what you're saying is that you can do this:
> 
> promise = ...
> promise.then(alpha).then(beta)
> promise.then(gamma).then(delta)
> 
> and in yield-coroutine style this is effectively:
> 
> value = yield promise
> beta(yield alpha(value))
> delta(yield gamma(value))
> 
> This deficiency is reasonably easy to work around with Deferreds.  You can just do:
> 
> def fork(d):
>     dprime = Deferred()
>     def propagate(result):
>         dprime.callback(result)
>         return result
>     d.addBoth(propagate)
>     return dprime
> 
> and then:
> 
> fork(x).addCallback(alpha).addCallback(beta)
> fork(x).addCallback(gamma).addCallback(delta)
> 
> Perhaps this function should be in Twisted; it's certainly come up a few times.
> 
> But, the fact that the original result is immediately forgotten can also be handy, because it helps the unused result get garbage collected faster, even if multiple things are hanging on to the Deferred after the initial result has been processed.  And it is actually pretty unusual to want to share the same result among multiple callers (which is why this function hasn't been added to the core yet).
> 
> -glyph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121016/b5e9c0cd/attachment.html>