Hi there, May I ask why you chose libaio over, for example, libevent? Arnar On Dec 20, 2007 2:42 AM, MichaĆ Pasternak <michal.dtz@gmail.com> wrote:
Hi,
I've spent last few days trying to write a wrapper for libaio.
Libaio is a simple Linux-only library, which should in theory support async read/write on a file descriptor. In practice - this depends on many strange requirements and the documentation is basically nonexistent (if you don't count lwn articles and kernel mailing list archives), so that last few days of C coding looked like this: (it's funny how every C coding session looks similar)
http://youtube.com/watch?v=-JbhMvzeX7s
Just to summarize... * no AIO reads from socket * not every filesystem supported * output buffer must be N-pages long and aligned to page start * file access is unbuffered (yep, no cache!) * ... and it may still - *silently* - block in some circumstances (like, when you run out of "block layer requests", like if I knew, what are those)
Anyways, libaio seems to be a cool idea - maybe someday kernel guys will do some more work on it (like, commit those patches which add buffer aio) and it will be possible to do AIO stuff on sockets. On the other hand, I found no way other to be informed about data availability, than periodically reaping events (want libaio and epoll integration? use some more kernel patches...)
You can get the module code and read about current linux aio problems here:
http://code.google.com/p/twisted-linux-aio/
It is not a general-purpose module - it's tightly tied to Twisted. Only reads are supported ATM and there's no Python queue, only that low level one (see TODO.txt). If you're lucky, you may even see the proof of concept code running - w/o blocking. There are more things to come, especially that async shutil.copyfileobj replacement... :)
Sad thing is, that Linux seems to lack much in this area, especially when compared to FreeBSD...
-- m
PS: I feel glad I didn't started with POSIX aio_* implementation - on Linux it is said to launch a new thread for every fd request! http://www.atnf.csiro.au/people/rgooch/linux/docs/io-events.html - way to go, glibc programmers! I hope things are looking better now, than they were that few years ago...
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python