Glyph, Ben, Amber,

So what’s the resolution on Future.__isfuture__ and fixing the
isinstance(obj, Future) checks from asyncio?

3.5.2 RC is only few days away, I can still make the change if
it’s a blocker for Twisted and Tornado.

Yury

> On Jun 6, 2016, at 5:35 PM, Glyph <glyph@twistedmatrix.com> wrote:
>
>>
>> On Jun 6, 2016, at 14:21, Guido van Rossum <guido@python.org> wrote:
>>
>> On Mon, Jun 6, 2016 at 1:54 PM, Glyph <glyph@twistedmatrix.com> wrote:
>>
>>
>>> On Jun 6, 2016, at 08:29, Guido van Rossum <guido@python.org> wrote:
>>>
>>> On Sun, Jun 5, 2016 at 10:16 PM, Glyph <glyph@twistedmatrix.com> wrote:
>>>>
>>>> On Jun 4, 2016, at 13:25, Ben Darnell <ben@bendarnell.com> wrote:
>>>>
>>>> If things are so sensitive to minor changes in timing, doesn't that set the
>>>> bar impossibly high for interoperability?
>>>>
>>>>
>>>> The sensitivity is not to changes in timing - i.e. when the wall-clock runs,
>>>> or ordering of non-deterministically ordered events - but rather to
>>>> reentrancy - whether certain things complete synchronously while the caller
>>>> is still on the stack and can depend on them having done so upon return.
>>>>
>>>> The recommended way of writing tests within Twisted these days depends
>>>> heavily on `.callback´ synchronously resolving a Deferred, which is what
>>>> adding a call_soon breaks.
>>>
>>> That's interesting, and also potentially worrisome (for interop, I'm
>>> not saying Twisted is wrong here).
>>>
>>> I think asyncio depends on the opposite: that if you add a callback to
>>> a Future that's ready it does *not* immediately run. Asyncio's promise
>>> is pretty strongly that callbacks are serialized (no callbacks running
>>> inside other callbacks). IIRC we experimented with other semantics and
>>> found that it was harder to reason about. (IMO if you *know* a Future
>>> is ready why add a callback to it rather than just calling the damn
>>> thing if that's what you want?)
>>
>> I don't think Twisted is necessarily right here either.  You're absolutely right that it's easier to reason about reentrancy in some cases if you have a might-be-fired-might-not Future vs. the same sort of Deferred.  I like the property where you can do:
>>
>> def test(test_case):
>>     a = asynchronously_something()
>>     test_case.this_must_not_have_a_result(a)
>>     cause_a_to_fire()
>>     test_case.assertEqual(test_case.this_must_have_a_result(a), something)
>>
>> but this is (somewhat) opposed to the fact that call_soon means you never get the nasty surprise where the callback added in the middle of a function gets run before the rest of it does.  So I think there are good properties in both cases and given some thought it is probably possible to map between them, but Deferred is lower-level here in the sense that it provides a way to do this both with the event loop and without.  You can always call_soon(deferred.callback) but you can't go the other way and force a Future to resolve synchronously - right?
>>
>> Right. I'm still unclear on what the compelling use case for that is (other than that Twisted has always done this). Is it performance? Is it callback ordering?
>
> Very, very early in the development of Deferreds, they worked the way Futures do; we changed it mainly to reduce coupling to the event loop so that we could test general-purpose algorithms (like gatherResults) without needing to spin an event loop to do it.  So the main use-case is testing.
>
>> I suppose Deferred has a method to mark it done.
>
> Yep; ".callback".
>
>> Does that immediately run the callbacks?
>
> It runs callbacks up to the point that the first one returns a Deferred, and then it waits for that one to be fired to continue running the chain.
>
> There is an exception here, where Deferred effectively opts in to Future-like behavior in a very specific case: if you are recursively giving results to a Deferred X that would un-block a Deferred Y inside a callback on Y, Y will not execute its own callbacks reentrantly; it waits until the current callback is done.
>
> So while the semantics of .callback() on a Deferred are clear-cut with respect to that Deferred itself, "continue any Deferreds waiting upon it" is a callback-like structure that is slightly squirrely in a very call_soon-like way to avoid surprise reentrancy and RecursionError explosions when having a structure like an asynchronous 'for' loop.
>
>> Or does it come in two flavors? Can a Deferred that's marked done ever revert back to being not done?
>
> No.  A Deferred that has been called back stays called back; callbacking it again is always an error.  However, it may pause running its chain if you return another Deferred in the middle someplace; the way to resume it is to give the inner Deferred a result; the outer one cannot be otherwise compelled to continue.
>
>> (I believe I once read the Deferred code enough to be able to find the answers, but I'm afraid I've never really needed what I learned then, so I've forgotten...)
>
> Happy to fill in these blanks; they're (mostly, modulo the weird exception for callbacks-in-callbacks) straightforward :).
>
> -glyph
>

> _______________________________________________
> Async-sig mailing list
> Async-sig@python.org
> https://mail.python.org/mailman/listinfo/async-sig
> Code of Conduct: https://www.python.org/psf/codeofconduct/

_______________________________________________
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/