[Python-ideas] Proposal: A simple protocol for generator tasks

[Greg Ewing]

Piet Delport wrote:

> 2. Each value yielded as a "step" represents a scheduling instruction,
>    or primitive, to be interpreted by the task's scheduler.

I don't think this technique should be used to communicate
with the scheduler, other than *maybe* for a *very* small
set of operations that are truly primitive -- and even then
I'm not convinced.

To begin with, there are some operations that *can't* rely
on yielded instructions as the only way of invoking them.
Spawning a task, for example -- there must be some way for
non-task code to invoke that, otherwise you wouldn't be able
to get top-level tasks into the system.

Also, consider the operation of unblocking a task that's
waiting for some event to occur. Often you will want to
invoke this using a callback from an event loop, which is
not a generator and can't yield anything to anywhere.

Given that these operations must provide a way of invoking
them using a plain function call, there is little reason
to provide a second way using a yielded instruction.

In any case, I believe that the public interface for *any*
scheduler operation should not be a yielded instruction,
but either a plain function or something called using
yield-from, for reasons I explained to Guido earlier.

> - Specialized sub-schedulers that run as a normal task within their
>   parent scheduler, but implement for example weighted or priority
>   queuing of their subtasks, or similar features.

There are problems with allowing multiple schedulers to
coexist within the one system, especially if yielded
instructions are the only way to communicate with them.

It might work for instructions to a task's own scheduler
concerning itself, but some operations need to operate on
a *different* task, e.g. unblocking a task when the event
it was waiting for occurs. How do you know which scheduler
is managing it? And even if you can find out, if you have
to control it using yielded instructions, you have no
way of yielding something to a different task's scheduler.

-- 
Greg