[Twisted-Python] Setting socket options before connect
I have a use case (namely setting the Linux-specific SO_MARK socket option) where I need to set the socket option after the socket object is created but before Twisted starts to try and connect. Is there a clean way to do this? Ideally there would be some sort of pre-connect function, similar to the preexec_fn for child processes, but any reasonably clean solution will do. (By clean I mean "doesn't have me duplicating hundreds of lines of Twisted code into my own Connector/Port/etc. classes" ;o)
Subclass twisted.internet.tcp.Client, override createInternetSocket() so it calls setsockopt() on the socket after you've called base implementation to create it. This breaks some abstraction boundaries, so it isn't great, but very little code duplication is involved. -- Itamar Turner-Trauring, Future Foundries LLC http://futurefoundries.com/ — Twisted consulting, training and support.
On 12/07/13 11:34, Itamar Turner-Trauring wrote:
Ah, ok. Presumably I also need to subclass Connector and override _makeTransport to use MyClient, then call MyConnector() directly (or subclass the reactor... shudder) Should there be something built in to Twisted for this? Should I open a ticket?
On Jul 12, 2013, at 5:09 AM, exarkun@twistedmatrix.com wrote:
Does such an API exist today, or should a ticket be filed for one? For everyone's information, in case it's not entirely clear from the documentation resources available: we hope to eventually deprecate the whole 'tcp' module so that people (myself included ;-)) stop subclassing stuff in it, so writing new code that depends on this, even the nominally "public" parts of the API (the bits without underscores) would be really unfortunate. If we can figure out something that uses totally public APIs without subclassing tcp.Client that would be best; if not, we should really have a ticket open to fix the API so that it is possible. -glyph
On 01:01 am, glyph@twistedmatrix.com wrote:
Hm, I'm not *totally* sure what you mean. There's the approach Itamar outlined, using APIs such as `twisted.internet.tcp.{Client,Server}`. I don't think we should codify this as the public, stable, encouraged API to use - for precisely the reasons you give below. There's various other APIs that are clearly related but definitely don't currently allow you to wedge this functionality in: 1) reactor.connectTCP - nowhere to pass extra socket options now, but we could add more arguments to it I suppose. Doesn't sound very nice to me. 2) endpoints? Again, no current support, but it's a place you could add new parameters. Of course, this isn't a complete solution, since endpoints mostly just use reactor methods to set things up - but if we had a nice endpoints-based API then we could have a gross lower-level API that no one actually has to use. Still, is "tcp:host=A:port=B:sockopt=TCP_CORK|TCP_QUICKACK" the road we want to go down? 3) More transport methods - but this is an incomplete solution, as certain sockopts only make sense before a connection, so once you have a transport it's too late. Maybe someone else has some suggestions from a totally different ballpark that solve the problem more pleasantly? Anyhow, I think this certainly means a ticket should be filed for introducing some API - but it seems that a little more discussion about what the API should be will still be necessary. Jean-Paul
On Jul 15, 2013, at 4:44 AM, exarkun@twistedmatrix.com wrote:
Mostly just socratic method: you said "don't do this, do something else", I am just curious what "do something else" is :-). Given that no documented, tested interfaces actually exist, "use a documented, tested interface" is not presently an option - "create a documented, tested interface" seems to be what you (well, we, really) are suggesting, which means somebody needs to come up with a specification and file a ticket to make the change to Twisted.
There's the approach Itamar outlined, using APIs such as `twisted.internet.tcp.{Client,Server}`. I don't think we should codify this as the public, stable, encouraged API to use - for precisely the reasons you give below.
Yup.
There's various other APIs that are clearly related but definitely don't currently allow you to wedge this functionality in:
1) reactor.connectTCP - nowhere to pass extra socket options now, but we could add more arguments to it I suppose. Doesn't sound very nice to me.
There are two general approaches I can think of here: 1. pass a callable object to be invoked on the ... socket? transport? ... before connect(). This seems problematic because it could violate lots of assumptions Twisted makes about the socket and create arbitrary I/O problems in the main loop, which won't be reported well. 2. pass a static description of things to do pre-connect. To me, 1. analogizes to 'preexec_fn' in subprocess vs. 2. posix_spawn static description of process-state options, which means I like 2. a lot better. So the question is: is there anything other than setting socket options might want to do pre-connect? If we can conclusively say not, then I think adding an argument to pass extra socket options seems fine. If so, then if we can enumerate the other things you might want to do, we could just add arguments for each of them.
2) endpoints? Again, no current support, but it's a place you could add new parameters. Of course, this isn't a complete solution, since endpoints mostly just use reactor methods to set things up - but if we had a nice endpoints-based API then we could have a gross lower-level API that no one actually has to use. Still, is "tcp:host=A:port=B:sockopt=TCP_CORK|TCP_QUICKACK" the road we want to go down?
This would still require support from connectTCP anyway, no? The sockopt:... key is just the, uh, "user interface" (for lack of a better word) for some structured functionality exposed at a lower level. So... that suggests that it just reverts to case 1.
3) More transport methods
That is already supported, in a sense, you can call '.getHandle()' and set socket options on it if you want to be just a little bit platform-specific. And we have things like setTcpNoDelay already, too, to avoid depending on python socket objects.
- but this is an incomplete solution, as certain sockopts only make sense before a connection, so once you have a transport it's too late.
Right, it's the timing that seems to be the issue.
Maybe someone else has some suggestions from a totally different ballpark that solve the problem more pleasantly?
Passing socket options to connectTCP is growing on me, but it sure would be nice to have something nicer.
Anyhow, I think this certainly means a ticket should be filed for introducing some API - but it seems that a little more discussion about what the API should be will still be necessary.
Agreed. But then, it's been a couple days and nobody else has contributed to this thread :-). -g
On 17/07/13 18:44, Glyph wrote:
Well, since I started it... Some kind of endpoint argument might be problematic for some use-cases. In particular, Twisted would have to know how to convert the argument into the value to pass into setsockopt() and possibly in a platform-specific way. The API as presented also omits the SOL. I guess it might be ok if there was a way to reliably inject unknown options with arbitrary payloads, but I'm struggling to see a clean way to do this with a "parse a string"-style API. So, vote me +0.5 for a "pre-connect" function. But but but... It might be possible to sidestep this entire issue by providing a clean way for an app to inject itself "above" the socket. I can think of a few use-cases for this, most notably things like cmsg/IP_HDRINCL which Twisted doesn't know about, and thus can't handle. So maybe the correct way to handle this is "implement your own FD object"?
On 10:27 am, p.mayers@imperial.ac.uk wrote:
It wasn't so much an API as a "Hey, I have an idea... endpoints... here is an example I can think of in 10 seconds." :)
I think you're talking about the fact that "sockopts" are random integers associated with other big piles of random integers. Some of them are flags you turn on, but some come with random payloads of basically no possible known shape. It sounds like you're trying to think of an API that will support any and all socket options without understanding them. This API exists already. It is `setsockopt`. There's no reason to re-invent it. I prefer the approach taken elsewhere in Twisted, where a particular option is given some consideration and an API that understands the option is introduced. This approach certainly has its shortcomings - for example, it doesn't support arbitrary options. :) Do people really like using `setsockopt` though?
Nothing stops anyone from doing this already, today. Except that it's a lot of work and no one seems to want any of these features badly enough to do it. Jean-Paul
On 18/07/13 13:10, exarkun@twistedmatrix.com wrote:
It wasn't so much an API as a "Hey, I have an idea... endpoints... here is an example I can think of in 10 seconds." :)
Understood!
Yep.
Sure.
No-one "likes" it, but if you need to set a socket option, you don't really have much choice (absent a higher-layer API).
Well, exactly. It's currently easier for me to sub-class a Twisted internal/private class. But if that goes away, that option is unavailable and a replacement would be nice - even if it's a convenience API like: connectExistingSocket(...) Isn't there code to do this as part of the systemd activation?
On 01:09 pm, p.mayers@imperial.ac.uk wrote:
Glad to hear it is universally reviled. ;) To be a bit more explicit, I think it's worth considering particular socket options individually and introducing high-level APIs for dealing with them. They don't get requested too often, so even though it's kind of labor intensive it still might not be overwhelming.
I really wanted to suggest something along those lines. The current code has the limitation that it will only accept an *already* connected socket. There isn't something that will accept a socket and then do the connection part for you. Maybe `connectExistingSocket` could still leverage this code somehow to simplify its implementation, though. Jean-Paul
On Thu, Jul 18, 2013 at 1:09 PM, <exarkun@twistedmatrix.com> wrote:
On 01:09 pm, p.mayers@imperial.ac.uk wrote:
...
Sorry it is not exactly about the subject, but what is the current API that accepts already connected socket? Some time ago I was looking for such an API without success (probably not very hard though). -- Mikhail Terekhov
On 01:35 am, termim@gmail.com wrote:
IReactorSocket.adoptStreamConnection was recently added: http://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.I... Jean-Paul
Subclass twisted.internet.tcp.Client, override createInternetSocket() so it calls setsockopt() on the socket after you've called base implementation to create it. This breaks some abstraction boundaries, so it isn't great, but very little code duplication is involved. -- Itamar Turner-Trauring, Future Foundries LLC http://futurefoundries.com/ — Twisted consulting, training and support.
On 12/07/13 11:34, Itamar Turner-Trauring wrote:
Ah, ok. Presumably I also need to subclass Connector and override _makeTransport to use MyClient, then call MyConnector() directly (or subclass the reactor... shudder) Should there be something built in to Twisted for this? Should I open a ticket?
On Jul 12, 2013, at 5:09 AM, exarkun@twistedmatrix.com wrote:
Does such an API exist today, or should a ticket be filed for one? For everyone's information, in case it's not entirely clear from the documentation resources available: we hope to eventually deprecate the whole 'tcp' module so that people (myself included ;-)) stop subclassing stuff in it, so writing new code that depends on this, even the nominally "public" parts of the API (the bits without underscores) would be really unfortunate. If we can figure out something that uses totally public APIs without subclassing tcp.Client that would be best; if not, we should really have a ticket open to fix the API so that it is possible. -glyph
On 01:01 am, glyph@twistedmatrix.com wrote:
Hm, I'm not *totally* sure what you mean. There's the approach Itamar outlined, using APIs such as `twisted.internet.tcp.{Client,Server}`. I don't think we should codify this as the public, stable, encouraged API to use - for precisely the reasons you give below. There's various other APIs that are clearly related but definitely don't currently allow you to wedge this functionality in: 1) reactor.connectTCP - nowhere to pass extra socket options now, but we could add more arguments to it I suppose. Doesn't sound very nice to me. 2) endpoints? Again, no current support, but it's a place you could add new parameters. Of course, this isn't a complete solution, since endpoints mostly just use reactor methods to set things up - but if we had a nice endpoints-based API then we could have a gross lower-level API that no one actually has to use. Still, is "tcp:host=A:port=B:sockopt=TCP_CORK|TCP_QUICKACK" the road we want to go down? 3) More transport methods - but this is an incomplete solution, as certain sockopts only make sense before a connection, so once you have a transport it's too late. Maybe someone else has some suggestions from a totally different ballpark that solve the problem more pleasantly? Anyhow, I think this certainly means a ticket should be filed for introducing some API - but it seems that a little more discussion about what the API should be will still be necessary. Jean-Paul
On Jul 15, 2013, at 4:44 AM, exarkun@twistedmatrix.com wrote:
Mostly just socratic method: you said "don't do this, do something else", I am just curious what "do something else" is :-). Given that no documented, tested interfaces actually exist, "use a documented, tested interface" is not presently an option - "create a documented, tested interface" seems to be what you (well, we, really) are suggesting, which means somebody needs to come up with a specification and file a ticket to make the change to Twisted.
There's the approach Itamar outlined, using APIs such as `twisted.internet.tcp.{Client,Server}`. I don't think we should codify this as the public, stable, encouraged API to use - for precisely the reasons you give below.
Yup.
There's various other APIs that are clearly related but definitely don't currently allow you to wedge this functionality in:
1) reactor.connectTCP - nowhere to pass extra socket options now, but we could add more arguments to it I suppose. Doesn't sound very nice to me.
There are two general approaches I can think of here: 1. pass a callable object to be invoked on the ... socket? transport? ... before connect(). This seems problematic because it could violate lots of assumptions Twisted makes about the socket and create arbitrary I/O problems in the main loop, which won't be reported well. 2. pass a static description of things to do pre-connect. To me, 1. analogizes to 'preexec_fn' in subprocess vs. 2. posix_spawn static description of process-state options, which means I like 2. a lot better. So the question is: is there anything other than setting socket options might want to do pre-connect? If we can conclusively say not, then I think adding an argument to pass extra socket options seems fine. If so, then if we can enumerate the other things you might want to do, we could just add arguments for each of them.
2) endpoints? Again, no current support, but it's a place you could add new parameters. Of course, this isn't a complete solution, since endpoints mostly just use reactor methods to set things up - but if we had a nice endpoints-based API then we could have a gross lower-level API that no one actually has to use. Still, is "tcp:host=A:port=B:sockopt=TCP_CORK|TCP_QUICKACK" the road we want to go down?
This would still require support from connectTCP anyway, no? The sockopt:... key is just the, uh, "user interface" (for lack of a better word) for some structured functionality exposed at a lower level. So... that suggests that it just reverts to case 1.
3) More transport methods
That is already supported, in a sense, you can call '.getHandle()' and set socket options on it if you want to be just a little bit platform-specific. And we have things like setTcpNoDelay already, too, to avoid depending on python socket objects.
- but this is an incomplete solution, as certain sockopts only make sense before a connection, so once you have a transport it's too late.
Right, it's the timing that seems to be the issue.
Maybe someone else has some suggestions from a totally different ballpark that solve the problem more pleasantly?
Passing socket options to connectTCP is growing on me, but it sure would be nice to have something nicer.
Anyhow, I think this certainly means a ticket should be filed for introducing some API - but it seems that a little more discussion about what the API should be will still be necessary.
Agreed. But then, it's been a couple days and nobody else has contributed to this thread :-). -g
On 17/07/13 18:44, Glyph wrote:
Well, since I started it... Some kind of endpoint argument might be problematic for some use-cases. In particular, Twisted would have to know how to convert the argument into the value to pass into setsockopt() and possibly in a platform-specific way. The API as presented also omits the SOL. I guess it might be ok if there was a way to reliably inject unknown options with arbitrary payloads, but I'm struggling to see a clean way to do this with a "parse a string"-style API. So, vote me +0.5 for a "pre-connect" function. But but but... It might be possible to sidestep this entire issue by providing a clean way for an app to inject itself "above" the socket. I can think of a few use-cases for this, most notably things like cmsg/IP_HDRINCL which Twisted doesn't know about, and thus can't handle. So maybe the correct way to handle this is "implement your own FD object"?
On 10:27 am, p.mayers@imperial.ac.uk wrote:
It wasn't so much an API as a "Hey, I have an idea... endpoints... here is an example I can think of in 10 seconds." :)
I think you're talking about the fact that "sockopts" are random integers associated with other big piles of random integers. Some of them are flags you turn on, but some come with random payloads of basically no possible known shape. It sounds like you're trying to think of an API that will support any and all socket options without understanding them. This API exists already. It is `setsockopt`. There's no reason to re-invent it. I prefer the approach taken elsewhere in Twisted, where a particular option is given some consideration and an API that understands the option is introduced. This approach certainly has its shortcomings - for example, it doesn't support arbitrary options. :) Do people really like using `setsockopt` though?
Nothing stops anyone from doing this already, today. Except that it's a lot of work and no one seems to want any of these features badly enough to do it. Jean-Paul
On 18/07/13 13:10, exarkun@twistedmatrix.com wrote:
It wasn't so much an API as a "Hey, I have an idea... endpoints... here is an example I can think of in 10 seconds." :)
Understood!
Yep.
Sure.
No-one "likes" it, but if you need to set a socket option, you don't really have much choice (absent a higher-layer API).
Well, exactly. It's currently easier for me to sub-class a Twisted internal/private class. But if that goes away, that option is unavailable and a replacement would be nice - even if it's a convenience API like: connectExistingSocket(...) Isn't there code to do this as part of the systemd activation?
On 01:09 pm, p.mayers@imperial.ac.uk wrote:
Glad to hear it is universally reviled. ;) To be a bit more explicit, I think it's worth considering particular socket options individually and introducing high-level APIs for dealing with them. They don't get requested too often, so even though it's kind of labor intensive it still might not be overwhelming.
I really wanted to suggest something along those lines. The current code has the limitation that it will only accept an *already* connected socket. There isn't something that will accept a socket and then do the connection part for you. Maybe `connectExistingSocket` could still leverage this code somehow to simplify its implementation, though. Jean-Paul
On Thu, Jul 18, 2013 at 1:09 PM, <exarkun@twistedmatrix.com> wrote:
On 01:09 pm, p.mayers@imperial.ac.uk wrote:
...
Sorry it is not exactly about the subject, but what is the current API that accepts already connected socket? Some time ago I was looking for such an API without success (probably not very hard though). -- Mikhail Terekhov
On 01:35 am, termim@gmail.com wrote:
IReactorSocket.adoptStreamConnection was recently added: http://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.I... Jean-Paul
participants (5)
-
exarkun＠twistedmatrix.com -
Glyph -
Itamar Turner-Trauring -
Mikhail Terekhov -
Phil Mayers