Guido wrote:

Brian,

(Also in response to your bug report.) I'm a little worried that upping the limit to 1024 would cause some performance problems if you're making a lot of select() calls. The select allocates three arrays of length FD_SETSIZE+3; each array item is 12 bytes. This is a total allocation of more than 36K for a meager select() call! And all that memory also has to be cleared by the FD_ZERO() call.

If you actually have that many sockets, that's worth paying for (the socket objects themselves use up just as much memory, and your Python data structures for the sockets, no matter how small, are probably several times bigger), but for a more typical program, I see this as a lot of overhead.

Is there a way that this can be done more dynamically, e.g. by making the set size as big as needed on windows but no bigger?

(Before you suggest allocating that memory statically, remember it's possible to call select from multiple threads. Allocating 36K of thread-local space for each thread also doesn't sound too pleasant.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

Hmm - after going through all of the Win32 sdks, it doesn't appear to be possible to do it any other way than as a -D option at compile time, so optimizing for the common case (folks who _don't_ need large numbers of fds) is reasonable. Since we distribute a python15.dll with Zope on windows, this isn't that big a deal for us - we can just compile in a higher limit in our distributed dll. I was mostly thinking of the win32 users who don't have the ability to rebuild their dll, but maybe this isn't that much of a problem; I suspect that the people who write significant socket apps that would run into this problem probably have access to a compiler if they need it. Brian Lloyd brian@digicool.com Software Engineer 540.371.6909 Digital Creations http://www.digicool.com