Hi Trent, On Wed, Nov 26, 2014 at 10:36 PM, Trent Nelson <trent@trent.me> wrote:

Have you seen this?:

https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploite...

I spend the first 80-ish slides on async I/O.

(That was a year ago. I've done 2-3 sprints on it since then and have gotten it to a point where I can back up the claims with hard numbers on load testing benchmarks, demonstrated in the most recent video: https://www.youtube.com/watch?v=4L4Ww3ROuro.)

Thanks. Have never seen it before. Do you have some detailed description about how I/O subsystem works? From what I see, pyparallel relies on offloading I/O operations to the IOCP. And it seems that I/O kernel which I'm describing can replace IOCP on unix. And allowing the C code to do more high-level stuff (like reconnecting and message delimitation), would be of a benefit for pyparallel too, as it would allow less heap allocations, smaller number of heap snasphot, etc. Still PyParallel, is much bigger departure from current Python. As it doesn't allow arbitrary code to run in threads, so it's much longer perspective, than I/O kernel which will be just a library on PyPI. -- Paul