[Python-ideas] asyncore: included batteries don't fit

[Greg Ewing]

Guido van Rossum wrote:
> Though I think you should not use
> 'thread' since that term is already reserved for OS threads as
> supported by the threading module. ... You could also use task, 
> which also doesn't have a core Python
> meaning. 
> 
> Also I think you can now revisit it and rewrite the code to use Python 3.3.

Both good ideas. I'll see about publishing an updated version.

> It does bother me somehow that you're not using .send() and yield
> arguments at all. I notice that you have a lot ofthree-line code
> blocks like this:
> 
>       block_for_reading(sock)
>       yield
>       data = sock.recv(1024)

I wouldn't say I have a "lot". In the spamserver, there are really
only three -- one for accepting a connection, one for reading from
a socket, and one for writing to a socket. These are primitive
operations that would be provided by an async socket library.

Generally, all the yields would be hidden inside primitives like
this. Normally, user code would never need to use 'yield', only
'yield from'.

This probably didn't come through as clearly as it might have in my
tutorial. Part of the reason is that at the time I wrote it, I was
having to manually expand yield-froms into for-loops, so I was
reluctant to use any more of them than I needed to. Also, yield-from
was a new and unfamiliar concept, and I didn't want to scare people
by overusing it. These considerations led me to push some of the
yields slightly further up the layer stack than they could be.

> 
> The general form seems to be:
> 
>       arrange for a callback when some operation can be done without blocking
>       yield
>       do the operation
> 
> This seems to be begging to be collapsed into a single line, e.g.
> 
>       data = yield sock.recv_async(1024)

I'm not sure how you're imagining that would work, but whatever
it is, it's wrong -- that just doesn't make sense.

What *would* make sense is

    data = yield from sock.recv_async(1024)

with sock.recv_async() being a primitive that encapsulates the
block/yield/process triplet.

> (I would also prefer to see the socket wrapped in an object that makes
> it hard to accidentally block.)

It would be straightforward to make the primitives be methods of a
socket wrapper object. I only used functions in the tutorial in the
interests of keeping the amount of machinery to a bare minimum.

> But surely there's still a place for send() and other PEP 342 features?

In the wider world of generator usage, yes. If you have a
generator that it makes sense to send() things into, for
example, and you want to factor part of it out into another
function, the fact that yield-from passes through sent values
is useful.

But we're talking about a very specialised use of generators
here, and so far I haven't thought of a use for sent or yielded
values in this context that can't be done in a more straightforward
way by other means.

Keep in mind that a value yielded by a generator being used as
part of a coroutine is *not* seen by code calling it with
yield-from. Rather, it comes out in the inner loop of the
scheduler, from the next() call being used to resume the
coroutine. Likewise, any send() call would have to be made
by the scheduler, not the yield-from caller.

So, the send/yield channel is exclusively for communication
with the *scheduler* and nothing else. Under the old way of
doing generator-based coroutines, this channel was used to
simulate a call stack by yielding 'call' and 'return'
instructions that the scheduler interpreted. But all that
is now taken care of by the yield-from mechanism, and there
is nothing left for the send/yield channel to do.

> my users sometimes want to
> treat something as a coroutine but they don't have any yields in it
> 
> def caller():
>   data = yield from reader()
> 
> def reader():
>     return 'dummy'
>     yield
> 
> works, but if you drop the yield it doesn't work. With a decorator I
> know how to make it work either way.

If you're talking about a decorator that turns a function
into a generator, I can't see anything particularly headachish
about that. If you mean something else, you'll have to elaborate.

-- 
Greg