[Python-ideas] Async API: some code to review

Yury Selivanov yselivanov.ml at gmail.com
Tue Oct 30 01:43:23 CET 2012


Guido,

Finally got some time to do a review & read what others posted.
Some comments are more general, some are more implementation-specific
(hopefully you want to hear latter ones as well)

And I'm still in the process of digesting your approach & code (as
I've spent too much time with my implementation)...

On 2012-10-28, at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
[...]
> polling.py: http://code.google.com/p/tulip/source/browse/polling.py
[...]

1. I'd make EventLoopMixin a separate entity from pollsters.  So that you'd
be able to add many different pollsters to one EventLoop.  This way
you can have specialized pollster for different types of IO, including
UI etc.

2. Sometimes, there is a need to run a coroutine in a threadpool.  I know it
sounds weird, but it's probably worth exploring.

3. In my framework each threadpool worker has its own local context, with
various information like what Task run the operation etc.

And few small things:

4. epoll.poll and other syscalls need to be wrapped in try..except to catch
and ignore (and log?) EINTR type of exceptions.

5. For epoll you probably want to check/(log?) EPOLLHUP and EPOLLERR errors 
too.

> scheduling.py: http://code.google.com/p/tulip/source/browse/scheduling.py
[...]

> In the docstrings I use the prefix "COROUTINE:" to indicate public
> APIs that should be invoked using yield from.
[...]

As others, I would definitely suggest adding a decorator to make
coroutines more distinguishable.  It would be even better if we can return
a tiny wrapper, that lets you to simply write 'doit.run().with_timeout(2.1)', 
instead of:

    task = scheduling.Task(doit(), timeout=2.1)
    task.start()
    scheduling.run()

And avoid manual Task instantiation at all.

I also liked the simplicity of the Task class.  I think it'd be easy
to mix greenlets in it by switching in a new greenlet on each 'step'.
That will give you 'yield_()' function, which you can use in the same
way you use 'yield' statement now (I'm not proposing to incorporate
greenlets in the lib itself, but rather to provide an option to do so)
Hence there should be a way to plug your own Task (sub-)class in.

Thank you,
Yury


More information about the Python-ideas mailing list