On Wed, Jun 8, 2016 at 1:52 PM, Yury Selivanov yselivanov@gmail.com wrote:
Glyph, Ben, Amber,
So what’s the resolution on Future.__isfuture__ and fixing the isinstance(obj, Future) checks from asyncio?
3.5.2 RC is only few days away, I can still make the change if it’s a blocker for Twisted and Tornado.
None of this is blocking Tornado - we shipped asyncio integration six months ago. There's some room for improvement, but I don't think there's a clear enough mandate to squeeze something in for this release. I'd rather take the time to sort out a more complete plan before the next release.
-Ben
Yury
On Jun 6, 2016, at 5:35 PM, Glyph glyph@twistedmatrix.com wrote:
On Jun 6, 2016, at 14:21, Guido van Rossum guido@python.org wrote:
On Mon, Jun 6, 2016 at 1:54 PM, Glyph glyph@twistedmatrix.com wrote:
On Jun 6, 2016, at 08:29, Guido van Rossum guido@python.org wrote:
On Sun, Jun 5, 2016 at 10:16 PM, Glyph glyph@twistedmatrix.com
wrote:
On Jun 4, 2016, at 13:25, Ben Darnell ben@bendarnell.com wrote:
If things are so sensitive to minor changes in timing, doesn't that
set the
bar impossibly high for interoperability?
The sensitivity is not to changes in timing - i.e. when the
wall-clock runs,
or ordering of non-deterministically ordered events - but rather to reentrancy - whether certain things complete synchronously while the
caller
is still on the stack and can depend on them having done so upon
return.
The recommended way of writing tests within Twisted these days depends heavily on `.callback´ synchronously resolving a Deferred, which is
what
adding a call_soon breaks.
That's interesting, and also potentially worrisome (for interop, I'm not saying Twisted is wrong here).
I think asyncio depends on the opposite: that if you add a callback to a Future that's ready it does *not* immediately run. Asyncio's promise is pretty strongly that callbacks are serialized (no callbacks running inside other callbacks). IIRC we experimented with other semantics and found that it was harder to reason about. (IMO if you *know* a Future is ready why add a callback to it rather than just calling the damn thing if that's what you want?)
I don't think Twisted is necessarily right here either. You're
absolutely right that it's easier to reason about reentrancy in some cases if you have a might-be-fired-might-not Future vs. the same sort of Deferred. I like the property where you can do:
def test(test_case): a = asynchronously_something() test_case.this_must_not_have_a_result(a) cause_a_to_fire() test_case.assertEqual(test_case.this_must_have_a_result(a),
something)
but this is (somewhat) opposed to the fact that call_soon means you
never get the nasty surprise where the callback added in the middle of a function gets run before the rest of it does. So I think there are good properties in both cases and given some thought it is probably possible to map between them, but Deferred is lower-level here in the sense that it provides a way to do this both with the event loop and without. You can always call_soon(deferred.callback) but you can't go the other way and force a Future to resolve synchronously - right?
Right. I'm still unclear on what the compelling use case for that is
(other than that Twisted has always done this). Is it performance? Is it callback ordering?
Very, very early in the development of Deferreds, they worked the way
Futures do; we changed it mainly to reduce coupling to the event loop so that we could test general-purpose algorithms (like gatherResults) without needing to spin an event loop to do it. So the main use-case is testing.
I suppose Deferred has a method to mark it done.
Yep; ".callback".
Does that immediately run the callbacks?
It runs callbacks up to the point that the first one returns a Deferred,
and then it waits for that one to be fired to continue running the chain.
There is an exception here, where Deferred effectively opts in to
Future-like behavior in a very specific case: if you are recursively giving results to a Deferred X that would un-block a Deferred Y inside a callback on Y, Y will not execute its own callbacks reentrantly; it waits until the current callback is done.
So while the semantics of .callback() on a Deferred are clear-cut with
respect to that Deferred itself, "continue any Deferreds waiting upon it" is a callback-like structure that is slightly squirrely in a very call_soon-like way to avoid surprise reentrancy and RecursionError explosions when having a structure like an asynchronous 'for' loop.
Or does it come in two flavors? Can a Deferred that's marked done ever
revert back to being not done?
No. A Deferred that has been called back stays called back; callbacking
it again is always an error. However, it may pause running its chain if you return another Deferred in the middle someplace; the way to resume it is to give the inner Deferred a result; the outer one cannot be otherwise compelled to continue.
(I believe I once read the Deferred code enough to be able to find the
answers, but I'm afraid I've never really needed what I learned then, so I've forgotten...)
Happy to fill in these blanks; they're (mostly, modulo the weird
exception for callbacks-in-callbacks) straightforward :).
-glyph
Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/