[Python-ideas] Async API

[Yury Selivanov]

Hello,

First of all, sorry for the lengthy email.  I've really tried to make it concise
and I hope that I didn't fail entirely.  At the beginning I want to describe
the framework my company has been working on for several years, and on which we
successfully deployed several web applications that are exposed to 1000s of users 
today.  It survived multiple architecture reviews and redesigns, so I believe its 
design is worth to be introduced here.

Secondly, I'm really glad that many advanced python developers find that use of
"yield" is viable for async API, and that it even may be "The Right Way".  Because 
when we started working on our framework that sounded nuts (and still sounds...)


The framework
=============

I'll describe here only the core functionality, not touching message bus & dispatch,
protocols design, IO layers, etc.  If someone gets interested - I can start another
thread.

The very core of the system is Scheduler.  I prefer it to be called "Scheduler",
and not "Reactor" or something else, because it's not just an event loop.  It loops 
over micro-threads, where a micro-thread is a primitive that holds a pointer to the
current running/suspended task.  Task can be anything, from coroutine, to a Sleep
command.  A Task may be suspended because of IO waiting, a lock primitive, a timeout
or something else.  You can even write programs that are not IO-bound at all.

To the code.  So when you have::

    @coroutine
    def foo():
        bar_value = yield bar()

defined, and then executed, 'foo' will send a Task object (wrapped around 'bar'), 
so that it will be executed in the foo's micro-thread.

And because we return a Task, we can also do::

    yield bar().with_timeout(1)

or even (alike coroutines with Futures)::

    bar_value_promise = yield bar().with_timeout(1).async()
    [some code]
    bar_value = yield bar_value_promise


So far there is nothing new.  The need for something "new" emerged when we started
to use it in "real world" applications.  Consider you have some ORM, and the 
following piece of code::

    topics = FE.select([
        FE.publication_date,
        FE.body,
        FE.category,
        (FE.creator, [
            (FE.creator.subject, [
                (gpi, [
                    gpi.avatar
                ])
            ])
        ])
    ]).filter(FE.publication_date < FE.publication_date.now(),
              FE.category == self.category)

and later::

    for topic in topics:
        print(topic.avatar.bucket.path, topic.category.name)

Everything is lazily-loaded, so a DB query here can be run at virtually any point.
When you iterate it pre-fetches objects, or addressing an attribute which wasn't
told to be loaded, etc.  The thing is that there is no way to express with 'yield'
all that semantics.  There is no 'for yield' statement, there is no pretty way of
resolving an attribute with 'yield'.

So even if you decide to write everything around you from scratch supporting
'yields', you still can't make a nice python API for some problems.

Another problem is that "writing everything from scratch" thing.  Nobody wants it.
We always want to reuse, nobody wants to write an SMTP client from scratch, when 
there is a decent one available right in the stdlib.

So the solution was simple.  Incorporate greenlets.

With greenlets we got a 'yield_()' function, that can be called from any coroutine,
and from framework user's point of view it is the same as 'yield' statement.

Now we were able to create a green-socket object, that looks as a plain stdlib
socket, and fix our ORM.  With it help we also were able to wrap lots and lots
of existing libraries in a nice 'yield'-style design, without rewriting their
internals.

At the end - we have a hybrid approach.  For 95% we use explicit 'yields', and for
the rest 5% - well, we know that when we use ORM it may do some implicit 'yields',
but that's OK.

Now, with adopting greenlets a whole new optimization set of strategies became
available.  For instance, we can substitute 'yield' statements with 'yield_'
command transparently by messing with opcodes, and by tweaking 'yield_' and
reusing 'Task' objects we can achieve near regular-python-call performance, but
with a tight control over our coroutines & micro-threads.  And when PyPy finally
adds support for Python 3, STM & JIT-able continulets, it would be very interesting
to see how we can improve performance even further.


Conclusion
==========

The whole point of this text was to show, that pure 'yield' approach will not
work.  Moreover, I don't think it's time to pronounce "The Right Way" of 'yielding'
and 'yield-fromming'.  There are so many ways of doing that: with @coroutine
decorator, plain generators, futures and Tasks, and perhaps more.  And I honestly
don't know with one is the right one.

What we really need now (and I think Guido has already mentioned that) is a 
callback-based (Deferreds, Futures, plain callbacks) design that is easy to 
plug-and-play in any coroutine-framework.  It has to be low-level and simple.  
Sort of WSGI for async frameworks ;)

We also need to work on the stdlib, so that it is easy to inject a custom socket 
in any object.  Ideally, by passing it in the constructor (as everybody hates 
monkey-patching.)

With all that said, I'd be happy to dedicate a fair amount of my time to help
with the design and implementation.


Thank you!
Yury