[Python-ideas] PEP 3156 EventLoop: hide details of iterations and idleness?

[Dustin J. Mitchell]

On Tue, Jan 22, 2013 at 2:19 PM, Guido van Rossum <guido at python.org> wrote:
> Ok, I'll kill call_every_iteration(). I'll wait for more discussion on
> run_once() and run()'s until-idle behavior.

One of the things that's been difficult for some time in Twisted is
writing clients in such a way that they reliably finish.  It's easy
for a simple client, but when the client involves several levels of
libraries doing mysterious, asynchronous things, it can be hard to
know when everything's really done.  Add error conditions in, and you
end up spending a lot of time thinking about something that, in a
synchronous program, is pretty simple.

One option, recently introduced to Twisted, is "react" -
http://twistedmatrix.com/documents/12.3.0/api/twisted.internet.task.html#react
 The idea is to encapsulate the lifetime of a client in a single
asynchronous operation; the synchronous parallel is libc calling
`exit` for you when `main` returns.  If all of your library code
cooperates and reliably indicates when it's done with any background
operations, then this is a good choice.

In cases where your libraries are less than perfect (perhaps they sync
to the cloud "in the background"), the run-until-idle behavior is
useful.  The client calls a function that triggers a cascade of
events.  When that cascade has exhausted itself, the process exits.
Synchronous, threaded programs do this with non-daemon threads.

I think that this option should be supported, if only for the
parallelism with synchronous code.


As for run-until-idle - I've used this sort of behavior occasionally
in tests, where I want to carefully control the sequence of
operations.  For example, I may want to reliably test handling of race
conditions:

op = start_operation()
while not in_critical_section():
    run_once()
generate_conflict()
while in_critical_section():
    run_once()
assert something()

Such a case would rely heavily on the details of the event loop.
Depending on how closely I want to tie my tests to that
implementation, that may or may not be OK.  If a particular event loop
implementation doesn't even *have* this model (as, it appears, Tornado
does not), then I think it would be fine to simply not implement this
operation.  So perhaps run_once() should be described as optional in
the PEP?

Dustin