Mailman 3 epoll implementation - Python-Dev

newer
PEP 42 - New kind of Temporary file

epoll implementation

older
Syntax errors on continuation lines

Ross Cohen

26 May 2006 26 May '06

4:34 p.m.

I wrote an epoll implementation which can be used as a drop-in replacement for parts of the select module (assuming the program is using only poll). The code can currently be used by doing: import epoll as select It was released under the Python license on sourceforge: http://sourceforge.net/projects/pyepoll Is there any interest in incorporating this into the standard python distribution? Ross

Show replies by date

skip＠pobox.com

26 May 26 May

4:47 p.m.

Ross> I wrote an epoll implementation which can be used as a drop-in Ross> replacement for parts of the select module ... Ross> Is there any interest in incorporating this into the standard Ross> python distribution? Without going to the trouble of downloading epoll (always an adventure with SourceForget), can you explain what epoll does, how it's better than (parts of) select, how widely it's used and how stable it is? Thx, Skip

Jean-Paul Calderone

5:10 p.m.

On Fri, 26 May 2006 11:47:43 -0500, skip@pobox.com wrote:

...

Ross> I wrote an epoll implementation which can be used as a drop-in Ross> replacement for parts of the select module ... Ross> Is there any interest in incorporating this into the standard Ross> python distribution?

Without going to the trouble of downloading epoll (always an adventure with SourceForget), can you explain what epoll does, how it's better than (parts of) select, how widely it's used and how stable it is?

epoll is a high-performance io notification mechanism provided by linux 2.6. It supports more sockets than select: it has no FD_SETSIZE or equivalent. It is more efficient than select: it scales roughly with the number of events instead of with the number of sockets. There is little or no controversy over its superiority to select and poll. The idea was first introduced to linux about five years ago, but the API has changed a lot in the interval. The epoll(4) API has been stable for all of linux 2.6. Including a wrapper for this functionality would be quite useful for many python network apps. However, I think with ctypes going into 2.5, it might be better to consider providing epoll support using a ctypes-based module rather than an extension module. Of course, if there is a volunteer to maintain and support an extension module, that's better than nothing. PyEpoll is missing a couple features I would like to see - the size of the epoll set is hard-coded to FD_SETSIZE, for example: while this makes little difference to Python 2.4.3 (since you cannot use sockets with fileno >= FD_SETSIZE at all in that version of Python), it is a serious limitation for other versions of Python. Similarly, the number of events to retrieve when invoking epoll_wait() is also hardcoded to FD_SETSIZE. Real applications often want to tune this value to avoid being overwhelmed by io events. These features could easily be added, I suspect, since they are primarily just extra integer parameters to various methods. Of course the other standard things should be added as well - documentation and test coverage, neither of which seem to be present at all in PyEpoll. Jean-Paul

Ross Cohen

5:31 p.m.

On Fri, May 26, 2006 at 01:10:30PM -0400, Jean-Paul Calderone wrote:

...

Including a wrapper for this functionality would be quite useful for many python network apps. However, I think with ctypes going into 2.5, it might be better to consider providing epoll support using a ctypes-based module rather than an extension module.

I have to admit that I wrote this for python 2.4 and haven't yet educated myself on ctypes. If this is what's desired I can put some energy into it. However, the PyEpoll module is largely a modified version of the python 2.4 select module.

...

Of course, if there is a volunteer to maintain and support an extension module, that's better than nothing. PyEpoll is missing a couple features I would like to see - the size of the epoll set is hard-coded to FD_SETSIZE, for example: while this makes little difference to Python 2.4.3 (since you cannot use sockets with fileno >= FD_SETSIZE at all in that version of Python), it is a serious limitation for other versions of Python. Similarly, the number of events to retrieve when invoking epoll_wait() is also hardcoded to FD_SETSIZE. Real applications often want to tune this value to avoid being overwhelmed by io events.

It is not true that this module limits the number of file descriptors to FD_SETSIZE. You were probable looking at the number of file descriptors returned by the epoll call, which is already limited by FD_SETSIZE because of the size of event array. In any case, this can be made tunable.

...

Of course the other standard things should be added as well - documentation and test coverage, neither of which seem to be present at all in PyEpoll.

Like I said, this code is a modified version of the python select module. It should (I hope, unless I accidentally removed some) include all the documentation from there, and any test cases for that module will apply here. The API provided to python programs has not changed, only the backend implementation of the API. My hope is that this can be included as a compile-time option for the select module instead of being its own module. Ross

Jean-Paul Calderone

6:57 p.m.

On Fri, 26 May 2006 13:31:33 -0400, Ross Cohen wrote:

...

On Fri, May 26, 2006 at 01:10:30PM -0400, Jean-Paul Calderone wrote:

...
Of course, if there is a volunteer to maintain and support an extension module, that's better than nothing. PyEpoll is missing a couple features I would like to see - the size of the epoll set is hard-coded to FD_SETSIZE, for example: while this makes little difference to Python 2.4.3 (since you cannot use sockets with fileno >= FD_SETSIZE at all in that version of Python), it is a serious limitation for other versions of Python. Similarly, the number of events to retrieve when invoking epoll_wait() is also hardcoded to FD_SETSIZE. Real applications often want to tune this value to avoid being overwhelmed by io events.

It is not true that this module limits the number of file descriptors to FD_SETSIZE. You were probable looking at the number of file descriptors returned by the epoll call, which is already limited by FD_SETSIZE because of the size of event array. In any case, this can be made tunable.

Woops, you're right, of course. I forgot that the argument to epoll_create is only a hint, not a limit. Sorry about that. Jean-Paul

Ross Cohen

5:13 p.m.

On Fri, May 26, 2006 at 11:47:43AM -0500, skip@pobox.com wrote:

...

Ross> I wrote an epoll implementation which can be used as a drop-in Ross> replacement for parts of the select module ... Ross> Is there any interest in incorporating this into the standard Ross> python distribution?

Without going to the trouble of downloading epoll (always an adventure with SourceForget), can you explain what epoll does, how it's better than (parts of) select, how widely it's used and how stable it is?

Sure, I can provide a short version of what's on this page: http://www.kegel.com/c10k.html epoll is a replacement for the standard poll() system call. It's available on Linux, introduced sometime during the 2.5 development series. Whereas poll requires that an fd list be maintained in userspace and handed in on every call, epoll maintains that list in the kernel and provides (de)registration functions for individual fds. This prevents the kernel from having to scan potentially thousands of file descriptors for each call to discover only a handful which had activity. Oddly enough, the interface to the poll call in the python select module much more closely resembles the design of epoll (kudos to whoever designed that). This code has undergone some testing with both codeville and BitTorrent, but I'm not going to claim it's seen as much testing as it should. The NeedForSpeed wiki lists /dev/epoll as a goal for the twisted folks. I assume that the naming here is either confusion with the Solaris /dev/poll or that it refers to the initial implementation of epoll, which I believe used a device file. Either way, this would give them that support for free, as well as everyone else. Ross

Sean Reifschneider

5:23 p.m.

On Fri, May 26, 2006 at 01:13:44PM -0400, Ross Cohen wrote:

...

The NeedForSpeed wiki lists /dev/epoll as a goal for the twisted folks. I

Just FYI, none of the Twisted items made it onto the list of items being worked. Thanks, Sean -- "You're thinking of Mr. Wizard." "[Emilio Lizardo's] a top scientist, dumbkopf." "So was Mr. Wizard." -- _Buckaroo_Banzai_ Sean Reifschneider, Member of Technical Staff tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability

Guido van Rossum

5:07 p.m.

I don't know what epoll is. On a related note, perhaps the needforspeed folks should look into supporting kqueue on systems where it's available? That's a really fast BSD-originated API to replace select/poll. (It's fast due to the way the API is designed.) --Guido On 5/26/06, Ross Cohen wrote:

...

I wrote an epoll implementation which can be used as a drop-in replacement for parts of the select module (assuming the program is using only poll). The code can currently be used by doing:

import epoll as select

It was released under the Python license on sourceforge: http://sourceforge.net/projects/pyepoll

Is there any interest in incorporating this into the standard python distribution?

Ross _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

Fredrik Lundh

5:28 p.m.

Guido van Rossum wrote:

...

I don't know what epoll is.

On a related note, perhaps the needforspeed folks should look into supporting kqueue on systems where it's available? That's a really fast BSD-originated API to replace select/poll. (It's fast due to the way the API is designed.)

roughly speaking, epoll is kqueue for linux. </F>

Greg Ewing

27 May 27 May

1:22 a.m.

Fredrik Lundh wrote:

...

roughly speaking, epoll is kqueue for linux.

There are many different select-like things around now (select, poll, epoll, kqueue -- are there others?) and random combinations of them seem to be available on any given platform. This makes writing platform-independent code that needs select-like functionality very awkward. Rather than adding yet another platform-dependent module, I'd like to see a unified Python interface in the stdlib that uses whichever is the best one available. -- Greg

Steve Holden

1:27 a.m.

Greg Ewing wrote:

...

Fredrik Lundh wrote:

...
roughly speaking, epoll is kqueue for linux.

There are many different select-like things around now (select, poll, epoll, kqueue -- are there others?) and random combinations of them seem to be available on any given platform. This makes writing platform-independent code that needs select-like functionality very awkward.

Rather than adding yet another platform-dependent module, I'd like to see a unified Python interface in the stdlib that uses whichever is the best one available.

Of course that would mean establishing which *was* the best available which, as we've seen this week, may not be easy. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Love me, love my blog http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden

Alex Martelli

3:59 a.m.

On May 26, 2006, at 6:27 PM, Steve Holden wrote:

...

Greg Ewing wrote:

...
Fredrik Lundh wrote:

...
roughly speaking, epoll is kqueue for linux.

There are many different select-like things around now (select, poll, epoll, kqueue -- are there others?) and random combinations of them seem to be available on any given platform. This makes writing platform-independent code that needs select-like functionality very awkward.

Rather than adding yet another platform-dependent module, I'd like to see a unified Python interface in the stdlib that uses whichever is the best one available.

Of course that would mean establishing which *was* the best available which, as we've seen this week, may not be easy.

I believe it's: kqueue on FreeBSD (for recent-enough versions thereof), otherwise epoll where available and nonbuggy, otherwise poll ditto, otherwise select -- that's roughly what Twisted uses for Reactor implementations, if memory serves me well. The platform- based heuristic should try to identify things this way but let the developer easily override if they know better. (One might add a Windows event loop as the best implementation available there -- Twisted does -- and GUI toolkit based event loops -- but in general that takes an abstraction level similar to Twisted's Reactor... maybe we should just repurpose that bit of Twisted?-) I don't think this is feasible for 2.5 (too late in the cycle to add major new stuff, IMHO), but it's well worth it for 2.6 (again IMHO). Alex

"Martin v. Löwis"

6:56 a.m.

Alex Martelli wrote:

...

...
Of course that would mean establishing which *was* the best available which, as we've seen this week, may not be easy.

I believe it's: kqueue on FreeBSD ...

Such a statement assumes they are semantically equivalent. However, I believe they aren't. A specific usage pattern can be expressed with any of them, for such a usage pattern, you can come up with a ranking even. For example, to check whether a single specific file descriptor is ready, I would assume that poll is best (even better than epoll and kqueue). To check whether file descriptor 0 is ready, I would assume that select is even better. Regards, Martin

Greg Ewing

7:53 a.m.

Alex Martelli wrote:

...

On May 26, 2006, at 6:27 PM, Steve Holden wrote:

...

...
Of course that would mean establishing which *was* the best available which, as we've seen this week, may not be easy.

I believe it's: kqueue on FreeBSD (for recent-enough versions thereof), otherwise epoll where available and nonbuggy, otherwise poll ditto, otherwise select

It would be an improvement if it would just pick *some* implementation that worked, even if it weren't strictly the best. -- Greg

Barry Scott

29 May 29 May

5:09 p.m.

On May 27, 2006, at 04:59, Alex Martelli wrote:

...

I believe it's: kqueue on FreeBSD (for recent-enough versions thereof), otherwise epoll where available and nonbuggy, otherwise poll ditto, otherwise select -- that's roughly what Twisted uses for

kqueue is not always faster. It depends on your application. The number of FDs and the fd values affect the point that kqueue wins over select. For a few FDs with low values select was always faster in my apps. Barry

Ganesan Rajagopal

27 May 27 May

5:13 a.m.

...

...
...
...
...
Steve Holden writes:

...

...
Rather than adding yet another platform-dependent module, I'd like to see a unified Python interface in the stdlib that uses whichever is the best one available.

Of course that would mean establishing which *was* the best available which, as we've seen this week, may not be easy.

There is a C aync event notification library called libevent which wraps kqueue, epoll, /dev/poll etc and provides a unified interface (See http://www.monkey.org/~provos/libevent/). Ganesan -- Ganesan Rajagopal

Ross Cohen

7:46 a.m.

On Sat, May 27, 2006 at 02:27:20AM +0100, Steve Holden wrote:

...

Greg Ewing wrote:

...
Rather than adding yet another platform-dependent module, I'd like to see a unified Python interface in the stdlib that uses whichever is the best one available.

Of course that would mean establishing which *was* the best available which, as we've seen this week, may not be easy.

Lots of systems (Windows? what's that?) implement both select and poll. I have yet to see a system which implements more than 1 of epoll, kqueue, /dev/poll, I/O completion ports, etc. You'd have to search long and hard for a case where the non-select/poll interfaces didn't perform better. Correctness/completeness is, of course, a stronger criterion than performance, so if kqueue on OS X doesn't support all relevant types of file descriptors, then it shouldn't used. Fortunately, the python select.poll interface is a good match for all of epoll, kqueue and /dev/poll. kqueue supports watching a lot more than file descriptors, but that doesn't matter much for this discussion. kqueue also mashes register/unregister/poll into a single call, which contributes to how confusing the interface is, though from what I can tell it really is basically the same as epoll. As for Windows, the WSASelect interface is truly awful. The python library currently sets it so you can pass in up to 512 descriptors, but don't count on getting even that many. There is a hard limit on the number of sockets a *user* can have open (varies based on OS version) with WSASelect and it will just start returning WSAENOBUFS once you go over. You then have the task of guessing how many you need to close before it will start working again. Firewalls especially can eat into the pool. IOCP has neither the WSAENOBUFS problem nor the 512 descriptor limit. It would be nice (if you or your customers run Windows) if someone did a select.poll interface with IOCP as a backend. Ross

"Martin v. Löwis"

6:51 a.m.

Greg Ewing wrote:

...

There are many different select-like things around now (select, poll, epoll, kqueue -- are there others?) and random combinations of them seem to be available on any given platform. This makes writing platform-independent code that needs select-like functionality very awkward.

Rather than adding yet another platform-dependent module, I'd like to see a unified Python interface in the stdlib that uses whichever is the best one available.

For epoll, that interface seems to be select.poll. Regards, Martin

Jonathan LaCour

26 May 26 May

5:57 p.m.

Guido van Rossum wrote:

...

On a related note, perhaps the needforspeed folks should look into supporting kqueue on systems where it's available? That's a really fast BSD-originated API to replace select/poll. (It's fast due to the way the API is designed.)

There is, in fact, an implementation of kqueue for Python already available. I have not used it myself, but it is available here: http://python-hpio.net/trac/ maybe the needforspeed people could take a look at this? -- Jonathan LaCour http://cleverdevil.org

"Martin v. Löwis"

6:42 p.m.

Jonathan LaCour wrote:

...

There is, in fact, an implementation of kqueue for Python already available. I have not used it myself, but it is available here:

http://python-hpio.net/trac/

maybe the needforspeed people could take a look at this?

"take a look" is really quite involved. We cannot just incorporate the code without permission of the author. So if the code is itself acceptable, then somebody would have to talk the author into contributing. Regards, Martin

Christopher Weimann

6:08 p.m.

On 05/26/2006-10:07AM, Guido van Rossum wrote:

...

I don't know what epoll is.

On a related note, perhaps the needforspeed folks should look into supporting kqueue on systems where it's available? That's a really fast BSD-originated API to replace select/poll. (It's fast due to the way the API is designed.)

http://python-hpio.net/trac/wiki/LibEventPython libevent-python is a CPython wrapper extension for Niels Provos' libevent, a small, fast asynchronous IO framework that supports select(), poll(), kqueue(), /dev/poll, and epoll(). Libevent can be found here http://monkey.org/~provos/libevent/ and it explains the advantages of epoll/kqueue over select/poll.

"Martin v. Löwis"

6:03 p.m.

Ross Cohen wrote:

...

Is there any interest in incorporating this into the standard python distribution?

I would like to see epoll support in Python, but not in the way PyEpoll is packaged. Instead, I think it should go into the select module, and be named epoll_*. Likewise, if kqueue was ever supported, I think it should also go into the select module. Regards, Martin

Ross Cohen

6:31 p.m.

On Fri, May 26, 2006 at 08:03:12PM +0200, "Martin v. Löwis" wrote:

...

Ross Cohen wrote:

...
Is there any interest in incorporating this into the standard python distribution?

I would like to see epoll support in Python, but not in the way PyEpoll is packaged. Instead, I think it should go into the select module, and be named epoll_*.

Likewise, if kqueue was ever supported, I think it should also go into the select module.

I agree that it should go into the select module, but not as a seperate set of calls. epoll is strictly better than poll. kqueue is strictly better than poll. Windows has its own completion ports API. Why should an application developer have to detect what platform they are running on? Why not simply provide the best implementation for the platform through the python API everyone is already using? Ross

"Martin v. Löwis"

6:48 p.m.

Ross Cohen wrote:

...

I agree that it should go into the select module, but not as a seperate set of calls. epoll is strictly better than poll. kqueue is strictly better than poll. Windows has its own completion ports API. Why should an application developer have to detect what platform they are running on?

Because the APIs have different semantics. Design some API for epoll, and make it replace select or poll (your choice), and I create you an application that breaks under your poll "emulation". If a complete emulation was possible, the OS developers would not have invented a new API.

...

Why not simply provide the best implementation for the platform through the python API everyone is already using?

Well, what *is* the API everyone is already using? This is really something for a higher-level API that assumes a certain processing model (unless the OS API, which assumes no processing model). Python has a tradition of exposing system APIs *as is*, with no second-guessing either the OS developers, nor the application developers. Then, we put a unified API *on top* of that. For event processing, the standard library always provided asyncore/asynchat, although people don't like it. I'm sure the Twisted people would integrate epoll in no time, and with no API change for *their* users. Regards, Martin

Ross Cohen

7:19 p.m.

On Fri, May 26, 2006 at 08:48:42PM +0200, "Martin v. Löwis" wrote:

...

Ross Cohen wrote:

...
I agree that it should go into the select module, but not as a seperate set of calls. epoll is strictly better than poll. kqueue is strictly better than poll. Windows has its own completion ports API. Why should an application developer have to detect what platform they are running on?

Because the APIs have different semantics. Design some API for epoll, and make it replace select or poll (your choice), and I create you an application that breaks under your poll "emulation".

It doesn't emulate the poll system call, it implements the poll call from the select module. See below.

...

If a complete emulation was possible, the OS developers would not have invented a new API.

True, and as I mentioned before, the python API more closely matches epoll in this case. The level-triggered mode of epoll is an almost perfect match. Someone went to some lengths to hide the details of the system poll interface.

...

...
Why not simply provide the best implementation for the platform through the python API everyone is already using?

Well, what *is* the API everyone is already using? This is really something for a higher-level API that assumes a certain processing model (unless the OS API, which assumes no processing model).

Without resorting to extension modules which are not part of the standard python library, people are using either select.poll or select.select. I was referring to select.poll, which is a better interface, but I believe isn't available on OS X.

...

Python has a tradition of exposing system APIs *as is*, with no second-guessing either the OS developers, nor the application developers. Then, we put a unified API *on top* of that. For event processing, the standard library always provided asyncore/asynchat, although people don't like it.

Someone already broke that tradition in this case. If the select.poll implementation is improved then everyone reaps the benefit. Ross

"Martin v. Löwis"

8:12 p.m.

Ross Cohen wrote:

...

True, and as I mentioned before, the python API more closely matches epoll in this case. The level-triggered mode of epoll is an almost perfect match. Someone went to some lengths to hide the details of the system poll interface.

Ah right, I missed that point. That makes it difficult to find an application that would break :-) One problem is that epoll allocates another file handle, when poll does not; the other problem could exist when the EPOLL constants differ in value from the POLL constants (which isn't the case on Linux), and then somebody uses numeric values instead of symbolic ones for register(). That said, I would be in favour of having select.poll "silently" use epoll where available. Of course, it would be good if a "cheap" run-time test could be made whether epoll is available at run-time (although just waiting for ENOSYS from epoll_create is probably cheap enough). Also, it would be good if the application could find out it is using epoll; for example, epollObject could expose a fileno member. Regards, Martin

Ross Cohen

8:34 p.m.

On Fri, May 26, 2006 at 10:12:15PM +0200, "Martin v. Löwis" wrote:

...

That said, I would be in favour of having select.poll "silently" use epoll where available. Of course, it would be good if a "cheap" run-time test could be made whether epoll is available at run-time (although just waiting for ENOSYS from epoll_create is probably cheap enough). Also, it would be good if the application could find out it is using epoll; for example, epollObject could expose a fileno member.

Woo! I'd be happy to code this up. First I'd like to get a feel for the prefered way of getting this integrated. Would people rather see: 1) Provide an epoll implementation which is used "silently" when the call is available. 2) Expose both poll(2) and epoll(4) in python and build select.poll on top of whatever is available. Ok, so 2 is only different in that it exposes the lower level APIs. I'd like to know if that's what people want. Ross

"Martin v. Löwis"

9:41 p.m.

Ross Cohen wrote:

...

1) Provide an epoll implementation which is used "silently" when the call is available.

2) Expose both poll(2) and epoll(4) in python and build select.poll on top of whatever is available.

Ok, so 2 is only different in that it exposes the lower level APIs. I'd like to know if that's what people want.

Supporting EPOLLET reliably looks like a valid use case, so there must be some way to specify "I really want epoll". Whether this is through a second entry point, or through an optional flag to poll, I don't care (providing the flag might be actually more expressive, since it would also allow to specify "I want poll", although I can't see a use case for that). Regards, Martin

Jean-Paul Calderone

6:49 p.m.

On Fri, 26 May 2006 14:31:33 -0400, Ross Cohen wrote:

...

On Fri, May 26, 2006 at 08:03:12PM +0200, "Martin v. Löwis" wrote:

...
Ross Cohen wrote:

...
Is there any interest in incorporating this into the standard python distribution?

I would like to see epoll support in Python, but not in the way PyEpoll is packaged. Instead, I think it should go into the select module, and be named epoll_*.

Likewise, if kqueue was ever supported, I think it should also go into the select module.

I agree that it should go into the select module, but not as a seperate set of calls.

How about "not *only* as a separate set of calls"? If poll(2) and epoll(4) are both available on the underlying platform, then they should both be exposed to Python as separate APIs. Then, on top of that, a relatively simple layer which selects the most efficient mechanism can be exposed, and developers can be encouraged to use that instead. Hiding the difference between poll and epoll would be detrimental to more advanced developers since it would hide the edge-triggered mode of epoll, which is still more efficient than the level-triggered mode, since only the level-triggered mode can be considered API-compatible with poll(2) without adding a much more complex layer on top.

...

epoll is strictly better than poll. kqueue is strictly better than poll.

AIUI, kqueue actually isn't implemented for PTYs on OS X, whereas poll(2) is. Given this, I don't think kqueue is actually strictly better. Although hopefully Apple will get their act together and fix this deficiency.

...

Windows has its own completion ports API. Why should an application developer have to detect what platform they are running on? Why not simply provide the best implementation for the platform through the python API everyone is already using?

So far as this is possible, I think it is a good idea. Jean-Paul

Ross Cohen

8:22 p.m.

On Fri, May 26, 2006 at 02:49:44PM -0400, Jean-Paul Calderone wrote:

...

On Fri, 26 May 2006 14:31:33 -0400, Ross Cohen wrote:

...
I agree that it should go into the select module, but not as a seperate set of calls.

How about "not *only* as a separate set of calls"? If poll(2) and epoll(4) are both available on the underlying platform, then they should both be exposed to Python as separate APIs. Then, on top of that, a relatively simple layer which selects the most efficient mechanism can be exposed, and developers can be encouraged to use that instead.

This is reasonable, though it requires some surgery to the select module. select.poll is a very reasonable layer over whatever mechanism is available. The system poll call would need to be properly wrapped and exposed in addition to epoll, because currently it is not.

...

Hiding the difference between poll and epoll would be detrimental to more advanced developers since it would hide the edge-triggered mode of epoll, which is still more efficient than the level-triggered mode, since only the level-triggered mode can be considered API-compatible with poll(2) without adding a much more complex layer on top.

I've never been fond of the edge-triggered mode. It's a pretty minor optimization, it saves scanning the set of previously returned fds to see if the events on them are still relevant. Given that there are added pitfalls to using that mode, it never seemed worth it. Any layer in python which tries to smooth over the differences will certainly kill the benefits. Of course, that's just my personal preference. If someone wants to use the edge-triggered mode, by all means let them.

...

...
epoll is strictly better than poll. kqueue is strictly better than poll.

AIUI, kqueue actually isn't implemented for PTYs on OS X, whereas poll(2) is. Given this, I don't think kqueue is actually strictly better. Although hopefully Apple will get their act together and fix this deficiency.

Ok, I'm not familiar with intimate details of kqueue. However, if there were a select.poll implementation based on kqueue, it would still be an improvement, since there isn't *any* implementation on OS X right now. Ross

Ronald Oussoren

9:43 p.m.

On 26-mei-2006, at 22:22, Ross Cohen wrote:

...

...
AIUI, kqueue actually isn't implemented for PTYs on OS X, whereas poll(2) is. Given this, I don't think kqueue is actually strictly better. Although hopefully Apple will get their act together and fix this deficiency.

Ok, I'm not familiar with intimate details of kqueue. However, if there were a select.poll implementation based on kqueue, it would still be an improvement, since there isn't *any* implementation on OS X right now.

Huh? osx support poll just fine, at least in recent versions. 10.3's poll is slightly broken, the one on 10.4 works fine (AFAIK). Ronald

Ross Cohen

10:46 p.m.

On Fri, May 26, 2006 at 11:43:05PM +0200, Ronald Oussoren wrote:

...

...
Ok, I'm not familiar with intimate details of kqueue. However, if there were a select.poll implementation based on kqueue, it would still be an improvement, since there isn't *any* implementation on OS X right now.

Huh? osx support poll just fine, at least in recent versions. 10.3's poll is slightly broken, the one on 10.4 works fine (AFAIK).

I'm probably not up to date. All I could find reference to was that poll() on 10.3 is emulated using select(). Ross

"Martin v. Löwis"

9:56 p.m.

Ross Cohen wrote:

...

I've never been fond of the edge-triggered mode. It's a pretty minor optimization, it saves scanning the set of previously returned fds to see if the events on them are still relevant. Given that there are added pitfalls to using that mode, it never seemed worth it. Any layer in python which tries to smooth over the differences will certainly kill the benefits.

I thought about this (even though I never used it), and I think there are good uses for EPOLLET. Suppose you have a large set of descriptors where you possibly want to write to. So you poll for write, then write, then poll again. Eventually you fill the write buffer, and need to wait for the other side to read some data; you block in poll. Later, you have sent all data you wanted to send, yet the connection remains open. So you have to *remove* the fd from the poll list, because otherwise poll would return immediately each time. So you keep adding and removing file descriptors to the epoll set, essentially for each write operation. I imagine it would be more efficient to just leave the write file descriptors in the set, even if you have sent all data. With EPOLLET, the epoll_wait will just ignore those descriptors, and it will watch only those for which you have actually filled the transmit queue, and where you have pending data. This feature sounds reasonable to me.

...

Ok, I'm not familiar with intimate details of kqueue. However, if there were a select.poll implementation based on kqueue, it would still be an improvement, since there isn't *any* implementation on OS X right now.

That depends on the kernel version, no? I though 10.4 added poll. Also, kqueue support would be for the BSDs primarily. If somebody contributed a patch, I would accept it providing it was disabled on OSX (assuming kqueue really is not true poll replacement on this system). Regards, Martin

Ross Cohen

11:23 p.m.

On Fri, May 26, 2006 at 11:56:06PM +0200, "Martin v. Löwis" wrote:

...

I thought about this (even though I never used it), and I think there are good uses for EPOLLET.

There are, if the programmer wants to deal with it. Easy enough to add the flag and give them the choice. I'll put together a select module which adds this and code and uses it if the right stuff is present at compile and run time. epoll also allows 64 bits of data to be tucked away and returned when events happen. Could be useful for saving a dict lookup for every event. However, there are some refcounting issues. Dict lookup per event could be traded for one on deregistration. All it needs is a small forward-compatible extension to the current select.poll API. Ross

"Martin v. Löwis"

27 May 27 May

6:36 a.m.

Ross Cohen wrote:

...

epoll also allows 64 bits of data to be tucked away and returned when events happen. Could be useful for saving a dict lookup for every event. However, there are some refcounting issues. Dict lookup per event could be traded for one on deregistration. All it needs is a small forward-compatible extension to the current select.poll API.

I don't understand. You only need the dictionary on registration, to find out whether this is a new registration or a modification of an existing one. How could you save the dict lookup at registration time? And what would you store in the additional data? Regards, Martin

Ross Cohen

7 a.m.

On Sat, May 27, 2006 at 08:36:12AM +0200, "Martin v. Löwis" wrote:

...

Ross Cohen wrote:

...
epoll also allows 64 bits of data to be tucked away and returned when events happen. Could be useful for saving a dict lookup for every event. However, there are some refcounting issues. Dict lookup per event could be traded for one on deregistration. All it needs is a small forward-compatible extension to the current select.poll API.

I don't understand. You only need the dictionary on registration, to find out whether this is a new registration or a modification of an existing one. How could you save the dict lookup at registration time? And what would you store in the additional data?

The first thing any user of the poll interface does with the file descriptor is map it to some state object. That's where the lookup can be saved, the object can just be handed back directly. Problem being that when the fd is unregistered, we won't get back the PyObject pointer so can't decrement the refcount, has to be stored and looked up manually. Ross

"Martin v. Löwis"

8 a.m.

Ross Cohen wrote:

...

The first thing any user of the poll interface does with the file descriptor is map it to some state object. That's where the lookup can be saved, the object can just be handed back directly. Problem being that when the fd is unregistered, we won't get back the PyObject pointer so can't decrement the refcount, has to be stored and looked up manually.

Ah. That would be an interface change; I would not do that. The lookup is usually-constant and very fast. It is complexity that grows with the number of file descriptors that these APIs worry about. Regards, Martin

"Martin v. Löwis"

26 May 26 May

8:27 p.m.

...

How about "not *only* as a separate set of calls"? If poll(2) and epoll(4) are both available on the underlying platform, then they should both be exposed to Python as separate APIs. Then, on top of that, a relatively simple layer which selects the most efficient mechanism can be exposed, and developers can be encouraged to use that instead.

Why does it have to be a separate API? Isn't the edge-triggered mode just a matter of passing EPOLLET into register? If so, I would advise against having two sets of functions, although having two sets of constants is propably fine. I would just provide a second constructor, select.epoll(), which guarantees that epoll is used as the implementation strategy (i.e. it would raise OSError if the kernel doesn't support epoll_create). Regards, Martin

6535

Age (days ago)

6538

Last active (days ago)

List overview

Download

37 comments

15 participants

participants (15)

"Martin v. Löwis"
Alex Martelli
Barry Scott
Christopher Weimann
Fredrik Lundh
Ganesan Rajagopal
Greg Ewing
Guido van Rossum
Jean-Paul Calderone
Jonathan LaCour
Ronald Oussoren
Ross Cohen
Sean Reifschneider
skip＠pobox.com
Steve Holden

epoll implementation

Sean Reifschneider

Ganesan Rajagopal

Jonathan LaCour

Christopher Weimann

tags

participants (15)