[Twisted-Python] WaitForMultipleObjects socket limitation
Hi, I've made a proof of concept for asynchronous console input on Windows [1] and now I am trying to understand the limits of WaitForMultipleObjects API I've used. Documentation on win32eventreactor mentions limit for 64 objects: http://twistedmatrix.com/documents/11.0.0/api/twisted.internet.win32eventrea... However, it is completely opaque what these objects are? For console handles and process handles it is quite obvious, but not for sockets. Is 64 the limit for total amount sockets opened on different ports? Is 64 the limit for connections made to a socket on specified port? 1. http://techtonik.rainforce.org/2011/03/asynchronous-input-from-windows-conso... -- anatoly t.
On Apr 7, 2011, at 12:16 PM, anatoly techtonik wrote:
I've made a proof of concept for asynchronous console input on Windows [1] and now I am trying to understand the limits of WaitForMultipleObjects API I've used.
Documentation on win32eventreactor mentions limit for 64 objects: http://twistedmatrix.com/documents/11.0.0/api/twisted.internet.win32eventrea... However, it is completely opaque what these objects are? For console handles and process handles it is quite obvious, but not for sockets.
Is 64 the limit for total amount sockets opened on different ports? Is 64 the limit for connections made to a socket on specified port?
1. http://techtonik.rainforce.org/2011/03/asynchronous-input-from-windows-conso...
64 is the limit for the total number of objects (listening ports, connections to a port, client connections, your console, the waker, serial ports, whatever) that WFMO may wait upon at once. Put another way, MAXIMUM_WAIT_OBJECTS=64: http://msdn.microsoft.com/en-us/library/ms687025(v=vs.85).aspx
On Thu, Apr 7, 2011 at 1:23 PM, Glyph Lefkowitz
On Apr 7, 2011, at 12:16 PM, anatoly techtonik wrote:
I've made a proof of concept for asynchronous console input on Windows [1] and now I am trying to understand the limits of WaitForMultipleObjects API I've used.
Documentation on win32eventreactor mentions limit for 64 objects:
http://twistedmatrix.com/documents/11.0.0/api/twisted.internet.win32eventrea...
However, it is completely opaque what these objects are? For console handles and process handles it is quite obvious, but not for sockets.
Is 64 the limit for total amount sockets opened on different ports? Is 64 the limit for connections made to a socket on specified port?
1. http://techtonik.rainforce.org/2011/03/asynchronous-input-from-windows-conso...
64 is the limit for the total number of objects (listening ports, connections to a port, client connections, your console, the waker, serial ports, whatever) that WFMO may wait upon at once.
Put another way, MAXIMUM_WAIT_OBJECTS=64: http://msdn.microsoft.com/en-us/library/ms687025(v=vs.85).aspx
Note that you can wait on more than 64 objects at a time, just not using a single WaitForMultipleObjects call. The MSDN page Glyph pointed out has a little more info. Kevin Horn
On Fri, Apr 8, 2011 at 4:52 AM, Kevin Horn
Note that you can wait on more than 64 objects at a time, just not using a single WaitForMultipleObjects call. The MSDN page Glyph pointed out has a little more info.
The proposed solutions, however, seem rather unsatisfactory. If you're going to start spawning new threads to monitor everything, you might as well just do IOCP in another thread, or even in the main thread (at least as far as I know). -- mithrandi, i Ainil en-Balandor, a faer Ambar
On Apr 8, 2011, at 2:56 AM, Tristan Seligmann wrote:
On Fri, Apr 8, 2011 at 4:52 AM, Kevin Horn
wrote: Note that you can wait on more than 64 objects at a time, just not using a single WaitForMultipleObjects call. The MSDN page Glyph pointed out has a little more info.
The proposed solutions, however, seem rather unsatisfactory. If you're going to start spawning new threads to monitor everything, you might as well just do IOCP in another thread, or even in the main thread (at least as far as I know).
I think we may be close to the point where we can drop win32eventreactor completely. I think IOCP can deal with arbitrary Windows events too, so if we just expose that in a compatible way, and whatever else comes along with it, we can just delete win32er without losing any functionality. (Or maybe we already do? Not my area of expertise any more :)).
On Fri, Apr 8, 2011 at 6:26 PM, Glyph Lefkowitz
On Apr 8, 2011, at 2:56 AM, Tristan Seligmann wrote:
On Fri, Apr 8, 2011 at 4:52 AM, Kevin Horn
wrote: Note that you can wait on more than 64 objects at a time, just not using a single WaitForMultipleObjects call. The MSDN page Glyph pointed out has a little more info.
The proposed solutions, however, seem rather unsatisfactory. If you're going to start spawning new threads to monitor everything, you might as well just do IOCP in another thread, or even in the main thread (at least as far as I know).
I think we may be close to the point where we can drop win32eventreactor completely. I think IOCP can deal with arbitrary Windows events too, so if we just expose that in a compatible way, and whatever else comes along with it, we can just delete win32er without losing any functionality. (Or maybe we already do? Not my area of expertise any more :)).
It's technically possible, but it's not a thing that iocpreactor currently does. Someone (hurr hurr) needs to stop slacking and implement it, along with support for waiting on an arbitrary number of handles. Also, how much do we care that win32er can be used without a compiler, but iocpreactor needs one to build the API wrapper?
On Sat, Apr 9, 2011 at 4:43 AM, Pavel Pergamenshchik
On Fri, Apr 8, 2011 at 6:26 PM, Glyph Lefkowitz
wrote: On Apr 8, 2011, at 2:56 AM, Tristan Seligmann wrote:
On Fri, Apr 8, 2011 at 4:52 AM, Kevin Horn
wrote: Note that you can wait on more than 64 objects at a time, just not using a single WaitForMultipleObjects call. The MSDN page Glyph pointed out has a little more info.
The proposed solutions, however, seem rather unsatisfactory. If you're going to start spawning new threads to monitor everything, you might as well just do IOCP in another thread, or even in the main thread (at least as far as I know).
I think we may be close to the point where we can drop win32eventreactor completely. I think IOCP can deal with arbitrary Windows events too, so if we just expose that in a compatible way, and whatever else comes along with it, we can just delete win32er without losing any functionality. (Or maybe we already do? Not my area of expertise any more :)).
It's technically possible, but it's not a thing that iocpreactor currently does. Someone (hurr hurr) needs to stop slacking and implement it, along with support for waiting on an arbitrary number of handles.
I've found a "tutorial" by Richard Tew. http://posted-stuff.blogspot.com/2009/07/iocp-based-sockets-with-ctypes-in_3... There is a lot of stuff to read. I can figure out if: 1. IOCP doesn't give a 100% CPU load when idle 2. It is doesn't seem like it is possible to listen to console events like WFMO, but I didn't try
Also, how much do we care that win32er can be used without a compiler, but iocpreactor needs one to build the API wrapper? -- anatoly t.
On Sat, Apr 9, 2011 at 3:43 AM, Pavel Pergamenshchik
Also, how much do we care that win32er can be used without a compiler, but iocpreactor needs one to build the API wrapper?
I suspect it's not terribly important, since Windows users can just use the binary distributions where somebody else has done all the hard work of compiling. -- mithrandi, i Ainil en-Balandor, a faer Ambar
participants (5)
-
anatoly techtonik
-
Glyph Lefkowitz
-
Kevin Horn
-
Pavel Pergamenshchik
-
Tristan Seligmann