[Python-ideas] An alternate approach to async IO

[Trent Nelson]

On Tue, Nov 27, 2012 at 05:09:17PM -0800, Sturla Molden wrote:
> 
> Den 28. nov. 2012 kl. 01:15 skrev Trent Nelson <trent at snakebite.org>:
> 
> > 
> >    Right, that's why I proposed using non-Python types as buffers
> >    whilst in the background IO threads.  Once the thread finishes
> >    processing the event, it pushes the necessary details onto a
> >    global interlocked list.  ("Details" being event type and
> >    possibly a data buffer if the event was 'data received'.)
> > 
> >    Then, when aio.events() is called, CPython code (holding the GIL)
> >    does an interlocked/atomic flush/pop_all, creates the relevant
> >    Python objects for the events, and returns them in a list for the
> >    calling code to iterate over.
> > 
> >    The rationale for all of this is that this approach should scale
> >    better when heavily loaded (i.e. tens of thousands of connections
> >    and/or Gb/s traffic).  When you're dealing with that sort of load
> >    on a many-core machine (let's say 16+ cores), an interlocked list
> >    is going to reduce latency versus 16+ threads constantly vying
> >    for the GIL.
> > 
> 
> Sorry. I changed my mind. I believe you are right after all :-)
> 
> I see two benefits:
> 
> 1. It avoids contention for the GIL and avoids excessive context
> shifts in the CPython interpreter.
> 
> 2. It potentially keeps the thread that runs the CPython interpreter
> in cache, as it is always active. And thus it also keeps the objects
> associated with the CPython interpreter in cache.
> 
> So yes, it might be better after all :-)

    That's even sweeter to read considering your initial opposition ;-)

> I don't think it would matter much for multicore scalability, as the
> Python processing is likely the more expensive part.

    Yeah, there will eventually be a point where the number of incoming
    events exceeds the number of events that can be processed by the
    single core event loop.  That's where the ideas for distributing
    event loop processing via multiprocessing come in.  That would be
    way down the track though -- and there is still lots of benefit
    to having the async background IO stuff and a single thread event
    loop.

        Trent.