[Python-ideas] Async API

[Guido van Rossum]

On Thu, Oct 25, 2012 at 11:39 AM, Yury Selivanov
<yselivanov.ml at gmail.com> wrote:
> Thank you for such a detailed and deep response.  Lots of good thoughts
> to digest.

You're welcome.

> One idea: the scope of the problem is enormously big.  It may take
> months/years to synchronize all ideas and thoughts by just communicating
> ideas over mail list without a concrete thing and subject to discuss.
> How about you/we create a repository with a draft implementation of
> scheduler/io loop/coroutines engine and we simply start tweaking an
> discussing that particular design?  That way people will see where
> to start the discussion, what's done, and some will even participate?
> The goal is not to write a production-quality software, but rather to
> have a common place to discuss/try things/benchmark etc.  I'm not sure,
> but maybe places like bitbucket, where you can have a wiki, issues, and
> the actual code is a better place, than a mail-list.

I am currently working on code. Steve Dower has also said he's going
to write some code. I'm just not quite ready to show my code (I need
to do a few more iterations on each component). As long as I can use
Mercurial I'm happy; bitbucket or Google Code Hosting both work fine
for me.

> I also think that there's need to move concurrency-related discussions
> to a separate mail-list, as everything else on python-ideas is lost
> now.

I don't have that problem. You are the one who started a new thread. :-)

If you really want a new mailing list, you can set it up; I'd be happy
to join, but my preference would be to stick it out here; I've seen
too many specialized lists and SIGs dwindle after an initial burst of
activity.

[...]
> The only problem I have with PEP-380, is that to me it's not entirely
> clear when you should use 'yield' or 'yield from' (please correct me if
> I am wrong).  I'll try to demonstrate it by example:
>
>
> class Socket:
>     def sendall(self, payload):
>         f = Future()
>         IOLoop.sendall(payload, future=f)
>         return f
>
> class SMTP:
>     def send(self, s):
>         ...
>         # yield the returned future to the scheduler
>         yield self.sock.sendall(s)
>         ...
>
> # And later:
> s = SMTP()
> yield from s.send('spam')
>
> Is it (roughly) how you want it all to look like?  I.e. using 'yield' to
> send a future/task to the scheduler, and 'yield from' to delegate?

I think that's the style that Steve Dower prefers. Greg Ewing would
rather see all public APIs use yield from, and reserve plain yield
exclusively as an implementation detail of the scheduler. In my own
experimental code I am using Greg's style and it is working out great.
My main reason for taking a hard stance on this is that it would
otherwise be too confusing for users -- should they use yield, yield
from, or a plain call? I'd like to tell them "if it blocks, use yield
from".

BTW, if you haven't read Greg's introduction to this style, here it is
-- worth reading!
http://www.cosc.canterbury.ac.nz/greg.ewing/python/tasks/SimpleScheduler.html

> If I guessed correctly, and that's how you envision it, I have a question:
> What if you decide to refactor 'Socket.sendall' to be a coroutine?
> In that case you'd want users to call it 'yield from Socket.sendall', and
> not 'yield Socket.sendall'.

That's why using yield from all the way is better!

-- 
--Guido van Rossum (python.org/~guido)