On 31 Mar, 10:15 pm, james@mansionfamily.plus.com wrote:
glyph@divmod.com wrote:
These implementations aren't "broken", so much as "limited". Well, if they don't honour what the documentation says, then they seem broken to me, and the documentation doesn't say you might get an unexpected 64 connection limit. No matter.
This is actually a problem for every reactor; handling of the connection limit (and there is a connection limit everywhere, even if it's 65535) is poorly documented and poorly handled. Patches which rectify this situation for any reactor, either from the perspective of docs or code, would of course be appreciated.
It still doesn't support SSL, though:
http://twistedmatrix.com/trac/ticket/593 Need something patterned very much like Len's stuff here http://www.lenholgate.com/archives/000456.html I guess.
The real solution to this problem is really: * Add an SSH IProtocol implementation that provides ITransport, rather than building it into the reactor * Add an API to ITransport to switch protocols, so that the protocol doesn't need to support protocol switching The BIO stuff mentioned on that URL is vaguely correct, but the interesting stuff is exposing it to Python. Now that JP Calderone has taken over maintenance of PyOpenSSL, we might actually have a chance to get that done :). Once you've got an SSL connection that is getting bytes from and delivering bytes to a transport, you don't need to care about any of the win32-specific aspects of that post; you just do the I/O using whatever platform I/O facility you want and the crypto happens completely disconnected from this socket. iocpreactor already implements enough of this to work. Again, if you want to see this happen: write code which does these things and contribute it.
Your suggestion for handle buckets in the win32eventreactor is one which has been suggested before and sounds good to me. Nobody has bothered to implement it yet :). However, even in its absence, I don't believe we'll be completely dropping win32eventreactor because it is useful for integrating with programs that do win32 drawing, even if iocpreactor has higher performance and is more scalable. Well, you need a daemon worker thread to call GetQueuedCompletionStatus(Ex)
(... blah blah win32 API stuff that I don't understand ...)
Why do you need an event reactor to support win32 applications, except to call MsgWait... rather than GetQueuedCompletionStatus?
My answer here, if I even have one, would be horribly vague: something about GDI+, or DirectX, or something. Maybe there's no good reason, I don't know. This level of specificity is probably best served by the issue tracker; file a ticket. And, if you're going to file a ticket, be prepared to actually follow up with an implementation. While we want to maintain support for Windows, the level of energy for doing really interesting Windows-specific stuff in Twisted is, in a word, "low". One thing you might want to know before you file that ticket is that *if* there's a reason for the way things are now, it's because in the Twisted idiom we always make sure GUI stuff and network stuff is happening in the same thread. If one of the approaches that you mentioned requires a dedicated "network" application thread, then that's probably why we aren't doing it.
On a similar sort of topic - is there a reason to have lots of implementations for POSIX, rather than use libevent or libev?
There is more than one answer to this question. Maybe someone will be helpful and turn some of these answers into a FAQ on the wiki: 1) Twisted predates libevent by a few years and libev by many years. One might instead ask why libevent didn't help us develop a C reactor, rather than writing a whole new library. 2) There's a libevent reactor in a branch. Contribute patches to deal with the outstanding review comments: http://twistedmatrix.com/trac/ticket/1930 3) Twisted runs on many platforms, so it supports the lowest common denominator; that is not libevent, it is Python's "select" module. 4) Neither libevent nor libev (to my knowledge) integrates with any GUI event loops. 5) Despite many valid rationalizations for its existence, the code in Twisted was developed organically over many years. The stuff you'll find here is the stuff that people thought was interesting and had time to work on. Strategically standardizing on a single low-level multiplexing mechanism is not something that is particularly fun or rewarding, especially when getting rid of the old code removes value for some users. (Not everyone already has libevent installed.)