On Sun, 6 Jan 2013 16:45:52 +0100 Markus <nepenthesdev@gmail.com> wrote:
Transports: I think SSL should be a Protocol not a transport - implemented using BIO pairs. If you can chain protocols, like Transport / ProtocolA / ProtocolB you can have TCP / SSL / HTTP as https or TCP / SSL / SOCKS / HTTP as https via ssl enabled socks proxy without having to much problems. Another example, shaping a connection TCP / RATELIMIT / HTTP.
Interesting idea. This may be up to the implementation -- not every implementation may have BIO wrappers available (AFAIK the stdlib doesn't),
Right, for ssl bios pyopenssl is required - or ctypes.
Or a patch to Python 3.4. See http://docs.python.org/devguide/ By the way, how does "SSL as a protocol" deal with SNI? How does the HTTP layer tell the SSL layer which servername to indicate? Or, on the server-side, how would the SSL layer invoke the HTTP layer's servername callback?
(I'm trying to follow Twisted's guidance here, they seem to have the longest experience doing these kinds of things. When I talked to Glyph IIRC he was skeptical about reconnecting in general.)
Point is - connections don't last forever, even if we want them to. If the transport supports "reconnect" - it is still upto the protocol to either support it or not. If a Protocol gets disconnected and wants to reconnect -without the Transport supporting .reconnect()- the protocol has to know it's factory.
+1 to this.
+ connection_established(self, transport) the connection is established - in your proposal it is connection_made which I disagree with due to the lack of context in the Futures, returns None
+ timeout_dns(self) Resolving the domain name failed - Protocol can .reconnect() for another try. returns None
+ timeout_idle(self) connection was idle for some time - send a high layer keep alive or close the connection - returns None
+ timeout_connecting(self) connection timed out connection - Protocol can .reconnect() for another try, returns None
I would rather have connection_failed(self, exc). (where exc can be a OSError or a socket.timeout)
* data_received(data) - if it was possible to return the number of bytes consumed by the protocol, and have the Transport buffer the rest for the next io in call, one would avoid having to do this in every Protocol on it's own - learned from experience.
Twisted has a whole slew of protocol implementation subclasses that implement various strategies like line-buffering (including a really complex version where you can turn the line buffering on and off) and "netstrings". I am trying to limit the PEP's size by not including these, but I fully expect that in practice a set of useful protocol implementations will be created that handles common cases. I'm not convinced that putting this in the transport/protocol interface will make user code less buggy: it seems easy for the user code to miscount the bytes or not return a count at all in a rarely taken code branch.
Please don't drop this.
You never know how much data you'll receive, you never know how much data you need for a message, so the Protocol needs a buffer. Having this io in buffer in the Transports allows every Protocol to benefit, they try to read a message from the data passed to data_received(), if the data received is not sufficient to create a full message, they need to buffer it and wait for more data.
Another solution for every Protocol to benefit is to provide a bunch of base Protocol implementations, as Twisted does: LineReceiver, etc. Your proposed solution (returning the number of consumed bytes) implies a lot of slicing and concatenation of immutable bytes objects inside the Transport, which may be quite inefficient. Regards Antoine.