On Fri, Jun 3, 2016 at 10:17 PM, Yury Selivanov <yselivanov@gmail.com> wrote:
> Actually, +1 for Ben’s suggestion to revisit if asyncio.Futures really need call_soon scheduling of callbacks. Specifically, a torrent of call_soon doesn’t really achieve “fair scheduling” as suggested by Yury anyway:
> https://gist.github.com/ambv/3324e3e569b9f7883dfcb9c8cb1d5445

One quick story about call_soon: while working on uvloop I once spent a couple of hours debugging why asyncio/sslproto.py didn’t work.  Turned out that SSLProtocol, in its connection_made scheduled some callbacks with loop.call_soon; and data_received did that too.  Turned out that libuv called data_received one loop iteration earlier, which resulted in a race condition.  What I’m trying to say here, is that making a subtle difference in how callbacks are scheduled can introduce some hard to debug bugs.

If things are so sensitive to minor changes in timing, doesn't that set the bar impossibly high for interoperability? In Tornado, we have similar concerns around "fair scheduling" of coroutines, but we solved this by using our equivalent of call_soon() in the coroutine runner rather than in the Future itself. So asyncio Futures yielded in a coroutine that uses Tornado's runner will be called one iteration later as they make two trips through the scheduler, while if asyncio.Task gained the ability to understand Tornado Futures they could turn into a busy loop that let nothing else run. If we want interoperability, we're going to have to mandate that code like SSLProtocol be tolerant of changes in timing (of course, saying that is one thing and achieving it is another)

In fact, I’m thinking now that Tornado/Twisted should keep their callbacks scheduling logic as is, and asyncio should keep its current implementation.

Yeah, the risks of making any changes in this area outweigh the benefits, since we'll never get all the implementations to schedule things in exactly the same way.


Anyways, I’ve tried to update asyncio Future to call callbacks directly:

The result is many broken unittests, until... a segfault in test_create_ssl_connection: https://gist.github.com/1st1/797dd12932bba7b353038fb7a80148ac (I’ll investigate this later).

I’d be -1 to change callbacks scheduling in 3.5, IMO this has to wait until 3.6 even if we decide this should be done.

> +1 for replacing isinstance checks with an attribute that Twisted can add to Deferreds.
> +1 for Ben’s suggestion that we should avoid ‘asyncio’ in the name here. __isfuture__?

Yes, an asyncio-agnostic name would be better.  I like __isfuture__.

Async-sig mailing list
Code of Conduct: https://www.python.org/psf/codeofconduct/