On Thu, 20 Dec 2007 09:23:43 +0000 "Arnar Birgisson" firstname.lastname@example.org wrote:
May I ask why you chose libaio over, for example, libevent?
libevent is to C as Twisted is to Python. Both run on many OS/platforms, but neither give you asynchronous file read/write; read(2) on a file descriptor (not socket) will be always blocking. At least this is what I understood after quick glance at libevent source - all it does in buffer.c:evbuffer_read on non-win32 OS is calling read(2).
libaio, instead, gives you async filesystem read/write access. Reading a few hundred megabytes of data won't block Twisted reactor, which is good.
Even better news is that since sending that e-mail yesterday I've found out how to integrate Linux async I/O with epoll notification, so the implementation I'm working on now will be even better. Pity it's Linux-only, but well... to be portable, it should be POSIX aio really, but I can't give you any guarantees about POSIX aio performance on Linux. I can try working on other OSs later, if there will be enough ppl interested.
Of course it would be cool to have Python libevent bindings - and even cooler it would be to have a portable libevent reactor in C - but would that really remove some of the limitations that Twisted has still to deal with? You'd still have to hack portable AIO implementation into libevent to have async file read/writes then ... :)