[Twisted-Python] What is the minimum effort solution to make inetd-managed twisted-based application?
What is the minimal effort method for building protocol instance (maybe out of already implemented protocol factory) using a transport, that uses parent-inherited sockets (or any other already connected sockets) ? I haven't yet found any single-line solution for that. For example, how to start an inetd-managed SMTP server (with STARTTLS support), suppose, we have protocol (and factory) already implemented, but how to construct the correct transport out of fd with the minimal effort? Or how to implement ucspi-tcp's tcpclient client on top of twisted framework? Right now I am looking at t.i.unix.Connector and t.i.unix.Port to understand how do transports get constructed by them, but well, that is too complex for a single evening. Should I really get into the details of implementing my own transport (or their constructors) to do what I need? I'm sure there should be something, that I missed in the documentation (or in the code?). And by the way, I haven't found any socketpair(2) usage in the twisted framework (except for tests), how can that be? Transport based on socketpair sockets will have the same implementation, as I need. Is it true, that nobody in twisted community uses anonymous preconnected sockets in real life? PS: I need socket-based transport, that is, full-duplex, half-closeable, with support of getting the remote endpoint address and with ability to start TLS on top of it and without implementing every that feature myself :) Thanks for a great framework! Alexey. -- с уважением, Алексей Шпагин системный администратор цеха передачи данных технического центра телекоммуникаций ОАО "ВолгаТелеком" филиал в Удмуртской республике.
On 12:57 pm, twisted-web@udmvt.ru wrote:
This isn't supported, but we'd like to support it. There's a ticket in the issue tracker, <http://twistedmatrix.com/trac/ticket/4387>, related to this (but with a somewhat wider scope).
Despite not appearing to be private, things like t.i.unix.Connector and t.i.unix.Port aren't really intended for use by applications. This is the right part of Twisted to be looking at if you want to contribute a patch which adds this feature, though. And I encourage you to do that. :) The implementation should be fairly straight-forward. Most things like Port and Client and Server have a "createInternetSocket" method. All that's really necessary to use an externally created file descriptor is get "createInternetSocket" to return that descriptor instead of creating a new one (or skip the call to the method entirely and just use the descriptor you have already). The biggest question I have is what the API should look like. Somehow the file descriptor needs to get from your application code (which knows that inetd put an open TCP connection on fd 0) to the Port/Client/Server. The obvious options, adding another argument to listen/connectTCP/UNIX/etc, would work, but is somewhat ugly (you have the issue that existing mandatory arguments would just be ignored). Another idea would be adding new methods entirely. I don't know if that's much better, though. So, if we can come up with a nice API, I think this will be a pretty quick feature to implement.
Apparently. :)
Hopefully if we can figure out how to create a Twisted transport object from an existing file descriptor, you should have no trouble with the rest of these.
Thanks for a great framework!
You're welcome, and thanks! :) Jean-Paul
On Tue, Jul 13, 2010 at 02:37:08PM -0000, exarkun@twistedmatrix.com wrote:
Oops, the Bad news.
Well, that feature will benefit from the feature I need. By the way, twisted architecture and abstractions allow one (for a limited number of protocols) to transfer the state along with the FD of the opened connection. Say, one may want to read HTTP headers and then send the connection FD to another process along with the read headers, so it will receive and "re-read" them with dataReceived(). Modification of HTTP protocol code will not be required here. But we need a modified transport constructor.
No, no, I'm only using them as a documentation about what transport looks like from the inside of the twisted, to other parts of the framework.
Well, I was trying to get into it during last evening, but it turned out to be more appropriate to spend a week or two on that process. It looks, like transport abstraction is not documented as a whole, only the user visible parts, or better say, protocol visible parts. I am still not sure whether it is advisable to subclass a base.BaseConnector or we can simply create a transport instance without that "complications". It looks like BaseConnector contain some portion of code, that should be reused.
That depends. Are we going to limit that feature to socket FDs only, or should we provide more specific interfaces for different type of files? Two (pipe) FDs may be used to construct full-duplex half-closeable transport too. Or may someone be interested also in master pty based transport? That would indeed require some terminal-specific support to be present in the transport instance, wouldn't it? Like transport.sendBreak() Perhaps one may use a full-duplex transport constructed from a tty/slave-pty FD? It may be useful to provide analogous interface for half-duplex connections too, IMHO. I think it may look like connection = somenewmodulename.socketConnection(fd, protocolFactory, addrFamily, sockType, reactor=None) or import socket connection = somenewmodulename.socketConnection(socket.fromfd(fd, addrFamily, sockType), protocolFactory, reactor=None) Create a transport object and associate/build a new protocol instance with/on it. For other FD types there should be different functions, returning instances of other types. Would you comment on that?
Adding new methods to reactor? I thought it can live completely separate from the reactor code and do not pollute reactor interface. Reactor is only used in the process of attaching a FD to a transport when new transport registers it's FDs in the reactor, so it can be an (optional?) argument to a constructor. However, yes, API consistency may suffer from that.
So, if we can come up with a nice API, I think this will be a pretty quick feature to implement.
I think it could be a set of functions inside a platform-dependent module if that functions will be FD-type dependent.
Oops, surprise!.. BTW creating a socketpair and forking and then communicating via PB seems like an elegant and convenient feature to me. Perhaps non-portable?
That's what I need, and I really hoped, that it is already implemented. What a surprise. Surely, I would not mind contibuting to such a great project, but first I would have to get more understanding of it's abstractions. Alexey. -- с уважением, Алексей Шпагин системный администратор цеха передачи данных технического центра телекоммуникаций ОАО "ВолгаТелеком" филиал в Удмуртской республике.
On 13 Jul, 04:52 pm, twisted-web@udmvt.ru wrote:
Ultimately it would be nice to support it for each transport type Twisted supports. That doesn't mean we have to do it all at once though. :)
Actually, in twisted/internet/process.py ("Do NOT use this module directly"), you'll see ProcessReader and ProcessWriter which are transports for pipes/ptys. The normal way to access these is via reactor.spawnProcess. But you'll notice that they even accept a fileno as an __init__ argument, unlike most other transports in Twisted. Mirroring this on the opposite side is twisted/internet/_posixstdio.py, which is a standard io transport, and also accepts filenos to its __init__.
Something to keep in mind here is that it may be desirable to support Windows at some point. This is one reason why it may make sense to have these APIs as methods on the reactor: the reactor can already easily be replaced by alternate implementations to cater to different platforms.
If it's possible to do this, it's probably a good idea. Just keep in mind that Windows doesn't use FDs for lots of things. StandardIO shows that the API inconsistency is annoying but not fatal. Plus, we're working on an API which can be a layer on top of this to make it all consistent again (twisted.internet.endpoints).
Windows doesn't have fork, so yea, you'll run into problems there. :) There are also problems with the way fork interacts with kqueue and epoll. The usual approach is reactor.spawnProcess to create an entirely new Python process, and then either connect to it over TCP or just use stdin/stdout to talk to it.
I'll answer whatever questions I can. :) Jean-Paul
On Wed, Jul 14, 2010 at 02:31:46AM -0000, exarkun@twistedmatrix.com wrote:
I'll answer whatever questions I can. :)
Oh, I have some questions... I asked some questions in the form of suggestions, well, don't take them seriously, I only ask, not really propose anything. However I'm ready to hear you about what I'm wrong with. Here, what I have discovered so far: There are transport factories in twisted, not defined as such, but really they are, like tcp.Port and tcp.Connector. They interface with protocol factories. That interface is asymmetric between server protocol factory and client protocol factory. Well, the implementation of client's side protocol factory and server's side should differ, but why should interfaces? That is okay, while you only have different and asymmetric transport factories the ones for client transports and the others for server transports. But is there really something in created and connected transport, that makes it server's or client's ? I guess nothing, except protocol instance attached. Q: Why should protocol factory interface "bother" about client/server dichotomy? For example, why should I be limited to only using PBClientFactory with connectXXX() variants of transport factories? Why should I do not want (under some crazy circumstances) to use reverse-connects and run PBServerFactory with connectXXX()? Well, maybe not PB, but what about other protocols? Should it work? Well, I suppose, it would break if you try now, but is is supposed to be that asymmetric? And after all, there is absolute symmetry in UDP-based transports. Q: Consider socketpair() variant - a pair of completely symmetric mutually connected sockets. Should I provide transport factory for use with server protocol factories and another transport factory for use with client protocol factories? That idea sounds stupid to me, there can be no difference in the implementation, except in the interfaces to protocol factories. I see there are some code, that can be moved from tcp.Port and tcp.Connector to some common base class of an abstract transport factory. That class could interfere with protocol factory without knowing whether it is server or client. Protocol factory already knows about it's own asymmetry. I do identify the events, that protocol factory receives from transport factory as these, please correct me if I'm wrong: global (factory-context) events: - start factory doStart - entering transport creation phase startedConnecting <server: when bind&listen succeeds why does anybody not invoke something like startedListenig ?> - stop factory doStop per transport connection events: - on transport creation <none exists?why?> <after connect() or listen() returns result, at this point we may decide to prevent further transport instantiation, especially if we are server-side protocol factory> - transport creation failed clientConnectionFailed <server: direct analogy - accept() may return an error, but that event not exists for server. why?> - transport created succesfully buildProtocol <a request to build a protocol instance implies, that transport instance have been succesfully created. or not?> - build protocol buildProtocol - after transport and protocol creation clientConnectionMade <in t.s.pb.PBXXXXFactory, event sent by protocol instance, name not defined in any Interface, why not?> - transport closed clientConnectionLost <why not define this for server too to be consistent?> They are not client/servers asymmetric as I see them, but have asymmetric names in twisted and sometimes when server variants exists, they have no fixed names defined in Interface classes :( What is the difference between doStart and startedConnecting (together with imaginary startedListening)? And who is responsible for sending which events? That is not defined in documentation. What do you think about these names: doStart() transportFactoryStarted() onBeforeTransportCreate() connectionFailed() buildProtocol() connectionMade() connectionLost() doStop() Well, I understand, you will veto them arguing by lot of code using old names, but I am interested to know will they break the abstraction of protocol factory? Can please anyone explain the asymmetry of the interfaces of protocol factories and tell how can it be useful for me. Thanks for your time. Alexey. -- с уважением, Алексей Шпагин системный администратор цеха передачи данных технического центра телекоммуникаций ОАО "ВолгаТелеком" филиал в Удмуртской республике.
On 14 Jul, 04:00 pm, twisted-web@udmvt.ru wrote:
startedConnecting, clientConnectionFailed, and clientConnectionLost only exist to facilitate client reconnection. See the end of the email for more about that.
That's probably true. And there could probably be a little less duplication of code between the Client and Server transports (although there's not a *lot* now).
You can certainly implement protocols that don't care about which side of the underlying transport is the client and which side is the server. It probably wouldn't be much work to get PB to be such an implementation. But apparently no one really cares about this, otherwise it might have come up before. ;)
And after all, there is absolute symmetry in UDP-based transports.
Okay. But SOCK_DGRAM is completely different from SOCK_STREAM, so I don't know if this comparison really means anything.
I'd probably start off with a low-level interface that's completely symmetric and doesn't involve factories. After all, it's not like socketpairs can spontaneously spring into existing (like client connections on a server can). On top of that, if you want, construct an API using factories to support multiple connections.
Make sure you don't get confused when you use "start factory" and "stop factory" as labels for doStart and doStop. The base implementation of doStart calls startFactory and the base implementation of doStop calls stopFactory. There's no startedListening because no one imagined there might be a kind of port that could only be created asynchronously. listenTCP, listenSSL, listenUNIX, listenUDP, listenMulticast can all complete immediately, so they all return an IPort provider. So if you wanted the startedListening callback, you can just call it yourself immediately after you call the listenXXX method. This may be an oversight for the general case. Vertex, for example, certainly wants to expose an asynchronous listen API. Other similar use-cases might be a listen API that uses UPnP to request a hole in a firewall. Still, listening is a one-off event. So if you can at least return a Deferred from your listen API, then when the Deferred fires, you have your "started listening" notification.
The factory can choose to do this in buildProtocol.
I don't think accept() failing is really analogous to clientConnectionFailed. Nevertheless, exposing accept() failures to application code is definitely a good idea. There's currently a ticket open for adding that feature.
In general, no such implication exists. It just means someone wants a new protocol instance to do something with. Usually that's because there's a new transport to connect it to, but maybe it's just a unit test calling the method.
PB wanted to put some logic on the factory when a new connection was first established. I'm not sure why it does this instead of putting the logic in Protocol.connectionMade.
- transport closed clientConnectionLost <why not define this for server too to be consistent?>
Consistency is alright, but sometimes it's not a sufficient justification. Over the years, I think the *existence* of ClientFactory.clientConnectionLost has caused more confusion and questions than the *lack* of ServerFactory.clientConnectionLost. In both cases, the information is available from Protocol.connectionLost.
doStart is called when a factory is hooked up to a network event source. That means there'll probably be just one call to doStart. For a reconnecting client factory, startedConnecting may be called many times. But really, doStart/startFactory/doStop/stopFactory are pretty obscure and rarely used. I wouldn't be surprised if we could come up with a better way to present this information pretty easily.
And who is responsible for sending which events? That is not defined in documentation.
The reactor implementation. There's no "how to implement a reactor" document because people don't do that very often.
What are they methods on? What are the precise semantics of each?
There's a lot of noise amongst the long-time Twisted developers about how client factories are no good and we should just dump them, leaving only server factories. That doesn't exactly make things symmetric, but it gets rid of the asymmetric factory interfaces. :)
After all, I'm posting my angry code. The first file is a module with some classes, that can be used to accomplish my task. The second file is a UNIX program, it uses socketpair() and then fork() to start both client and server, they talk each other via PB protocol. Running that produces some noise to the screen and then they exit. That was my minimal effort... That is really a partial solution to the problem, but: a. it already works for me, b. I'm not sure it will work for other protocols, c. I found it too complex and involving a lot of other code to make it in a proper way so, sorry, I haven't worked toward a real contribution to a twisted code. Perhaps anyone may suggest another "proper" solution, more elegant? I'm interested in having this kind of functionality supported by twisted. -- Alexey.
On Wed, 2010-07-28 at 15:14 +0400, twisted-web@udmvt.ru wrote:
You probably want to use reactor.spawnProcess, not fork - it also allows you to pass arbitrary file descriptors to the child process: http://twistedmatrix.com/documents/current/core/howto/process.html Also, an example: http://twistedmatrix.com/trac/browser/tags/releases/twisted-10.1.0/twisted/r...
On Wed, Jul 28, 2010 at 08:10:48AM -0400, Itamar Turner-Trauring wrote:
Also, an example: http://twistedmatrix.com/trac/browser/tags/releases/twisted-10.1.0/twisted/r...
Thanks.
-- Alexey.
On 12:57 pm, twisted-web@udmvt.ru wrote:
This isn't supported, but we'd like to support it. There's a ticket in the issue tracker, <http://twistedmatrix.com/trac/ticket/4387>, related to this (but with a somewhat wider scope).
Despite not appearing to be private, things like t.i.unix.Connector and t.i.unix.Port aren't really intended for use by applications. This is the right part of Twisted to be looking at if you want to contribute a patch which adds this feature, though. And I encourage you to do that. :) The implementation should be fairly straight-forward. Most things like Port and Client and Server have a "createInternetSocket" method. All that's really necessary to use an externally created file descriptor is get "createInternetSocket" to return that descriptor instead of creating a new one (or skip the call to the method entirely and just use the descriptor you have already). The biggest question I have is what the API should look like. Somehow the file descriptor needs to get from your application code (which knows that inetd put an open TCP connection on fd 0) to the Port/Client/Server. The obvious options, adding another argument to listen/connectTCP/UNIX/etc, would work, but is somewhat ugly (you have the issue that existing mandatory arguments would just be ignored). Another idea would be adding new methods entirely. I don't know if that's much better, though. So, if we can come up with a nice API, I think this will be a pretty quick feature to implement.
Apparently. :)
Hopefully if we can figure out how to create a Twisted transport object from an existing file descriptor, you should have no trouble with the rest of these.
Thanks for a great framework!
You're welcome, and thanks! :) Jean-Paul
On Tue, Jul 13, 2010 at 02:37:08PM -0000, exarkun@twistedmatrix.com wrote:
Oops, the Bad news.
Well, that feature will benefit from the feature I need. By the way, twisted architecture and abstractions allow one (for a limited number of protocols) to transfer the state along with the FD of the opened connection. Say, one may want to read HTTP headers and then send the connection FD to another process along with the read headers, so it will receive and "re-read" them with dataReceived(). Modification of HTTP protocol code will not be required here. But we need a modified transport constructor.
No, no, I'm only using them as a documentation about what transport looks like from the inside of the twisted, to other parts of the framework.
Well, I was trying to get into it during last evening, but it turned out to be more appropriate to spend a week or two on that process. It looks, like transport abstraction is not documented as a whole, only the user visible parts, or better say, protocol visible parts. I am still not sure whether it is advisable to subclass a base.BaseConnector or we can simply create a transport instance without that "complications". It looks like BaseConnector contain some portion of code, that should be reused.
That depends. Are we going to limit that feature to socket FDs only, or should we provide more specific interfaces for different type of files? Two (pipe) FDs may be used to construct full-duplex half-closeable transport too. Or may someone be interested also in master pty based transport? That would indeed require some terminal-specific support to be present in the transport instance, wouldn't it? Like transport.sendBreak() Perhaps one may use a full-duplex transport constructed from a tty/slave-pty FD? It may be useful to provide analogous interface for half-duplex connections too, IMHO. I think it may look like connection = somenewmodulename.socketConnection(fd, protocolFactory, addrFamily, sockType, reactor=None) or import socket connection = somenewmodulename.socketConnection(socket.fromfd(fd, addrFamily, sockType), protocolFactory, reactor=None) Create a transport object and associate/build a new protocol instance with/on it. For other FD types there should be different functions, returning instances of other types. Would you comment on that?
Adding new methods to reactor? I thought it can live completely separate from the reactor code and do not pollute reactor interface. Reactor is only used in the process of attaching a FD to a transport when new transport registers it's FDs in the reactor, so it can be an (optional?) argument to a constructor. However, yes, API consistency may suffer from that.
So, if we can come up with a nice API, I think this will be a pretty quick feature to implement.
I think it could be a set of functions inside a platform-dependent module if that functions will be FD-type dependent.
Oops, surprise!.. BTW creating a socketpair and forking and then communicating via PB seems like an elegant and convenient feature to me. Perhaps non-portable?
That's what I need, and I really hoped, that it is already implemented. What a surprise. Surely, I would not mind contibuting to such a great project, but first I would have to get more understanding of it's abstractions. Alexey. -- с уважением, Алексей Шпагин системный администратор цеха передачи данных технического центра телекоммуникаций ОАО "ВолгаТелеком" филиал в Удмуртской республике.
On 13 Jul, 04:52 pm, twisted-web@udmvt.ru wrote:
Ultimately it would be nice to support it for each transport type Twisted supports. That doesn't mean we have to do it all at once though. :)
Actually, in twisted/internet/process.py ("Do NOT use this module directly"), you'll see ProcessReader and ProcessWriter which are transports for pipes/ptys. The normal way to access these is via reactor.spawnProcess. But you'll notice that they even accept a fileno as an __init__ argument, unlike most other transports in Twisted. Mirroring this on the opposite side is twisted/internet/_posixstdio.py, which is a standard io transport, and also accepts filenos to its __init__.
Something to keep in mind here is that it may be desirable to support Windows at some point. This is one reason why it may make sense to have these APIs as methods on the reactor: the reactor can already easily be replaced by alternate implementations to cater to different platforms.
If it's possible to do this, it's probably a good idea. Just keep in mind that Windows doesn't use FDs for lots of things. StandardIO shows that the API inconsistency is annoying but not fatal. Plus, we're working on an API which can be a layer on top of this to make it all consistent again (twisted.internet.endpoints).
Windows doesn't have fork, so yea, you'll run into problems there. :) There are also problems with the way fork interacts with kqueue and epoll. The usual approach is reactor.spawnProcess to create an entirely new Python process, and then either connect to it over TCP or just use stdin/stdout to talk to it.
I'll answer whatever questions I can. :) Jean-Paul
On Wed, Jul 14, 2010 at 02:31:46AM -0000, exarkun@twistedmatrix.com wrote:
I'll answer whatever questions I can. :)
Oh, I have some questions... I asked some questions in the form of suggestions, well, don't take them seriously, I only ask, not really propose anything. However I'm ready to hear you about what I'm wrong with. Here, what I have discovered so far: There are transport factories in twisted, not defined as such, but really they are, like tcp.Port and tcp.Connector. They interface with protocol factories. That interface is asymmetric between server protocol factory and client protocol factory. Well, the implementation of client's side protocol factory and server's side should differ, but why should interfaces? That is okay, while you only have different and asymmetric transport factories the ones for client transports and the others for server transports. But is there really something in created and connected transport, that makes it server's or client's ? I guess nothing, except protocol instance attached. Q: Why should protocol factory interface "bother" about client/server dichotomy? For example, why should I be limited to only using PBClientFactory with connectXXX() variants of transport factories? Why should I do not want (under some crazy circumstances) to use reverse-connects and run PBServerFactory with connectXXX()? Well, maybe not PB, but what about other protocols? Should it work? Well, I suppose, it would break if you try now, but is is supposed to be that asymmetric? And after all, there is absolute symmetry in UDP-based transports. Q: Consider socketpair() variant - a pair of completely symmetric mutually connected sockets. Should I provide transport factory for use with server protocol factories and another transport factory for use with client protocol factories? That idea sounds stupid to me, there can be no difference in the implementation, except in the interfaces to protocol factories. I see there are some code, that can be moved from tcp.Port and tcp.Connector to some common base class of an abstract transport factory. That class could interfere with protocol factory without knowing whether it is server or client. Protocol factory already knows about it's own asymmetry. I do identify the events, that protocol factory receives from transport factory as these, please correct me if I'm wrong: global (factory-context) events: - start factory doStart - entering transport creation phase startedConnecting <server: when bind&listen succeeds why does anybody not invoke something like startedListenig ?> - stop factory doStop per transport connection events: - on transport creation <none exists?why?> <after connect() or listen() returns result, at this point we may decide to prevent further transport instantiation, especially if we are server-side protocol factory> - transport creation failed clientConnectionFailed <server: direct analogy - accept() may return an error, but that event not exists for server. why?> - transport created succesfully buildProtocol <a request to build a protocol instance implies, that transport instance have been succesfully created. or not?> - build protocol buildProtocol - after transport and protocol creation clientConnectionMade <in t.s.pb.PBXXXXFactory, event sent by protocol instance, name not defined in any Interface, why not?> - transport closed clientConnectionLost <why not define this for server too to be consistent?> They are not client/servers asymmetric as I see them, but have asymmetric names in twisted and sometimes when server variants exists, they have no fixed names defined in Interface classes :( What is the difference between doStart and startedConnecting (together with imaginary startedListening)? And who is responsible for sending which events? That is not defined in documentation. What do you think about these names: doStart() transportFactoryStarted() onBeforeTransportCreate() connectionFailed() buildProtocol() connectionMade() connectionLost() doStop() Well, I understand, you will veto them arguing by lot of code using old names, but I am interested to know will they break the abstraction of protocol factory? Can please anyone explain the asymmetry of the interfaces of protocol factories and tell how can it be useful for me. Thanks for your time. Alexey. -- с уважением, Алексей Шпагин системный администратор цеха передачи данных технического центра телекоммуникаций ОАО "ВолгаТелеком" филиал в Удмуртской республике.
On 14 Jul, 04:00 pm, twisted-web@udmvt.ru wrote:
startedConnecting, clientConnectionFailed, and clientConnectionLost only exist to facilitate client reconnection. See the end of the email for more about that.
That's probably true. And there could probably be a little less duplication of code between the Client and Server transports (although there's not a *lot* now).
You can certainly implement protocols that don't care about which side of the underlying transport is the client and which side is the server. It probably wouldn't be much work to get PB to be such an implementation. But apparently no one really cares about this, otherwise it might have come up before. ;)
And after all, there is absolute symmetry in UDP-based transports.
Okay. But SOCK_DGRAM is completely different from SOCK_STREAM, so I don't know if this comparison really means anything.
I'd probably start off with a low-level interface that's completely symmetric and doesn't involve factories. After all, it's not like socketpairs can spontaneously spring into existing (like client connections on a server can). On top of that, if you want, construct an API using factories to support multiple connections.
Make sure you don't get confused when you use "start factory" and "stop factory" as labels for doStart and doStop. The base implementation of doStart calls startFactory and the base implementation of doStop calls stopFactory. There's no startedListening because no one imagined there might be a kind of port that could only be created asynchronously. listenTCP, listenSSL, listenUNIX, listenUDP, listenMulticast can all complete immediately, so they all return an IPort provider. So if you wanted the startedListening callback, you can just call it yourself immediately after you call the listenXXX method. This may be an oversight for the general case. Vertex, for example, certainly wants to expose an asynchronous listen API. Other similar use-cases might be a listen API that uses UPnP to request a hole in a firewall. Still, listening is a one-off event. So if you can at least return a Deferred from your listen API, then when the Deferred fires, you have your "started listening" notification.
The factory can choose to do this in buildProtocol.
I don't think accept() failing is really analogous to clientConnectionFailed. Nevertheless, exposing accept() failures to application code is definitely a good idea. There's currently a ticket open for adding that feature.
In general, no such implication exists. It just means someone wants a new protocol instance to do something with. Usually that's because there's a new transport to connect it to, but maybe it's just a unit test calling the method.
PB wanted to put some logic on the factory when a new connection was first established. I'm not sure why it does this instead of putting the logic in Protocol.connectionMade.
- transport closed clientConnectionLost <why not define this for server too to be consistent?>
Consistency is alright, but sometimes it's not a sufficient justification. Over the years, I think the *existence* of ClientFactory.clientConnectionLost has caused more confusion and questions than the *lack* of ServerFactory.clientConnectionLost. In both cases, the information is available from Protocol.connectionLost.
doStart is called when a factory is hooked up to a network event source. That means there'll probably be just one call to doStart. For a reconnecting client factory, startedConnecting may be called many times. But really, doStart/startFactory/doStop/stopFactory are pretty obscure and rarely used. I wouldn't be surprised if we could come up with a better way to present this information pretty easily.
And who is responsible for sending which events? That is not defined in documentation.
The reactor implementation. There's no "how to implement a reactor" document because people don't do that very often.
What are they methods on? What are the precise semantics of each?
There's a lot of noise amongst the long-time Twisted developers about how client factories are no good and we should just dump them, leaving only server factories. That doesn't exactly make things symmetric, but it gets rid of the asymmetric factory interfaces. :)
After all, I'm posting my angry code. The first file is a module with some classes, that can be used to accomplish my task. The second file is a UNIX program, it uses socketpair() and then fork() to start both client and server, they talk each other via PB protocol. Running that produces some noise to the screen and then they exit. That was my minimal effort... That is really a partial solution to the problem, but: a. it already works for me, b. I'm not sure it will work for other protocols, c. I found it too complex and involving a lot of other code to make it in a proper way so, sorry, I haven't worked toward a real contribution to a twisted code. Perhaps anyone may suggest another "proper" solution, more elegant? I'm interested in having this kind of functionality supported by twisted. -- Alexey.
On Wed, 2010-07-28 at 15:14 +0400, twisted-web@udmvt.ru wrote:
You probably want to use reactor.spawnProcess, not fork - it also allows you to pass arbitrary file descriptors to the child process: http://twistedmatrix.com/documents/current/core/howto/process.html Also, an example: http://twistedmatrix.com/trac/browser/tags/releases/twisted-10.1.0/twisted/r...
On Wed, Jul 28, 2010 at 08:10:48AM -0400, Itamar Turner-Trauring wrote:
Also, an example: http://twistedmatrix.com/trac/browser/tags/releases/twisted-10.1.0/twisted/r...
Thanks.
-- Alexey.
participants (4)
-
Alexey -
exarkun＠twistedmatrix.com -
Itamar Turner-Trauring -
twisted-web＠udmvt.ru