epoll() and kqueue wrapper for the select module
Linux Kernel 2.6+ and BSD (including Mac OS X) have two I/O event notification systems similar but superior to select() and poll(). From the manuals: kqueue, kevent -- kernel event notification mechanism The kqueue() system call provides a generic method of notifying the user when an event happens or a condition holds, based on the results of small pieces of kernel code termed filters. A kevent is identified by the (ident, filter) pair; there may only be one unique kevent per kqueue. epoll - I/O event notification facility epoll is a variant of poll(2) that can be used either as an edge-triggered or a level-triggered interface and scales well to large numbers of watched file descriptors. I've written wrappers for both mechanisms. Both wrappers are inspired from Twisted and select.poll()'s API. The interface is more Pythonic than the available wrappers and it reduced the burden on the user. The users don't have to deal with low level control file descriptors and the fd is closed automatically when the object is collected. epoll interface
ep = select.epoll(1) ep.register(fd, select.EPOLL_IN | select.EPOLL_OUT) ep.modify(fd, select.EPOLL_OUT) events = ep.wait(1, 1000) ep.unregister(fd)
kqueue interface The kqueue interface is more low level than the epoll interface. It has too many options.
kq = select.kqueue() ev = [select.kevent(fd, select.KQ_FILTER_WRITE, select.KQ_EV_ONESHOT | select.KQ_EV_ADD)] kq.control(ev, 0, 0) events = kq.control([], 1, None)
I already talked to some Twisted guys and they really like it. A patch is available at http://bugs.python.org/issue1657. The code needs more unit tests and documentation updates. Christian
On Thu, Dec 20, 2007 at 06:08:47PM +0100, Christian Heimes wrote:
I've written wrappers for both mechanisms. Both wrappers are inspired from Twisted and select.poll()'s API. The interface is more Pythonic than the available wrappers and it reduced the burden on the user. The users don't have to deal with low level control file descriptors and the fd is closed automatically when the object is collected.
Did you look at the python-epoll module which has been in the Cheese Shop for quite some time? There is no messing with a low level control file descriptor and it presents an identical interface to select.poll(). I would claim this whole new interface:
epoll interface
ep = select.epoll(1) ep.register(fd, select.EPOLL_IN | select.EPOLL_OUT) ep.modify(fd, select.EPOLL_OUT) events = ep.wait(1, 1000) ep.unregister(fd)
Is a greater burden on the user than:
import epoll as select
I would also claim that adding a new interface to accomplish the same task is not more pythonic. But I did write the python-epoll module in question, so I may be a bit biased. Ross
On 12/20/07, Ross Cohen
On Thu, Dec 20, 2007 at 06:08:47PM +0100, Christian Heimes wrote:
I've written wrappers for both mechanisms. Both wrappers are inspired from Twisted and select.poll()'s API. The interface is more Pythonic than the available wrappers and it reduced the burden on the user. The users don't have to deal with low level control file descriptors and the fd is closed automatically when the object is collected.
Did you look at the python-epoll module which has been in the Cheese Shop for quite some time? There is no messing with a low level control file descriptor and it presents an identical interface to select.poll().
I would claim this whole new interface:
epoll interface
ep = select.epoll(1) ep.register(fd, select.EPOLL_IN | select.EPOLL_OUT) ep.modify(fd, select.EPOLL_OUT) events = ep.wait(1, 1000) ep.unregister(fd)
Is a greater burden on the user than:
import epoll as select
I would also claim that adding a new interface to accomplish the same task is not more pythonic. But I did write the python-epoll module in question, so I may be a bit biased.
Ross
Providing a compatibility layer that mirrors the select.select or select.poll interface using the best underlying syscall may be nice but... In order to gain the most benefit from any of these system calls they should be exposed directly. Do the compatibility wrapping in python but leave the low level calls exposed for those that want them, they make a big difference to what an application can do. (i believe kqueue/kevent can handle signals, etc) I like this patch.
Ross Cohen wrote:
Did you look at the python-epoll module which has been in the Cheese Shop for quite some time? There is no messing with a low level control file descriptor and it presents an identical interface to select.poll().
No, I didn't see the module. To be honest I didn't look at the Python Package Index but used Google to search for epoll wrappers. I found the wrapper in Twisted on my disk and two other wrappers. One was written in pure C and the other one used ctypes. Your wrapper is a good implementation. I even found an inconvenience in my implementation when I studied your code. My wrapper raised an exception when a closed fd was removed with EPOLL_CTL_DEL.
I would also claim that adding a new interface to accomplish the same task is not more pythonic. But I did write the python-epoll module in question, so I may be a bit biased.
I agree with Gregory on that part of your answer. Christian
This makes me very happy.
I could try to work on integrating this stuff in asyncore, if someone
finds this of some interest.
On 20 Dic, 18:08, Christian Heimes
Linux Kernel 2.6+ and BSD (including Mac OS X) have two I/O event notification systems similar but superior to select() and poll(). From the manuals:
kqueue, kevent -- kernel event notification mechanism
The kqueue() system call provides a generic method of notifying the user when an event happens or a condition holds, based on the results of small pieces of kernel code termed filters. A kevent is identified by the (ident, filter) pair; there may only be one unique kevent per kqueue.
epoll - I/O event notification facility
epoll is a variant of poll(2) that can be used either as an edge-triggered or a level-triggered interface and scales well to large numbers of watched file descriptors.
I've written wrappers for both mechanisms. Both wrappers are inspired from Twisted and select.poll()'s API. The interface is more Pythonic than the available wrappers and it reduced the burden on the user. The users don't have to deal with low level control file descriptors and the fd is closed automatically when the object is collected.
epoll interface
ep = select.epoll(1) ep.register(fd, select.EPOLL_IN | select.EPOLL_OUT) ep.modify(fd, select.EPOLL_OUT) events = ep.wait(1, 1000) ep.unregister(fd)
kqueue interface
The kqueue interface is more low level than the epoll interface. It has too many options.>>> kq = select.kqueue()
ev = [select.kevent(fd, select.KQ_FILTER_WRITE,
select.KQ_EV_ONESHOT | select.KQ_EV_ADD)]
kq.control(ev, 0, 0) events = kq.control([], 1, None)
I already talked to some Twisted guys and they really like it. A patch is available athttp://bugs.python.org/issue1657. The code needs more unit tests and documentation updates.
Christian
_______________________________________________ Python-Dev mailing list Python-...@python.orghttp://mail.python.org/mailman/listinfo/python-dev Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...
participants (4)
-
Christian Heimes -
Giampaolo Rodola' -
Gregory P. Smith -
Ross Cohen