[Twisted-Python] Perspective broker and pipes
Hello,
I wrote a pb application which also uses udp sockets. It works nicely. Now there's the situation where client and server are both located on the same machine. This means I could just connect to localhost and everything works. Unfortunately some firewalls warn about connections. I want to avoid this. So I thought about replacing the transports in pb with pipes instead of sockets if the server and client are both on the same machine. I'd have to do the same for my custom udp solution. Doing it for my custom udp code is easy, but how do I do this with pb? Do I just have to create some kind of reactor.createPipe() thing or is there more to it?
Thanks for your help, -Matthias
On Wed, 31 Oct 2007 08:41:10 +0100, Nitro nitro@dr-code.org wrote:
Hello,
I wrote a pb application which also uses udp sockets. It works nicely. Now there's the situation where client and server are both located on the same machine. This means I could just connect to localhost and everything works. Unfortunately some firewalls warn about connections. I want to avoid this. So I thought about replacing the transports in pb with pipes instead of sockets if the server and client are both on the same machine. I'd have to do the same for my custom udp solution. Doing it for my custom udp code is easy, but how do I do this with pb? Do I just have to create some kind of reactor.createPipe() thing or is there more to it?
Since you're concerned about firewalls warning about connections, I'm guessing you're thinking about Windows. In that case, you can't use IReactorUNIX and IReactorUNIXDatagram, which would perhaps meet your use case (although for all I know, some firewall software warns about UNIX connections).
There is no support for communicating over arbitrary pipes in Twisted. There is incidental support for communicating with pipes connected to another process created with reactor.spawnProcess, but the process API is not the same as the IProtocol API.
In general, using pipes will lessen performance anyway.
Jean-Paul
Am 31.10.2007, 12:59 Uhr, schrieb Jean-Paul Calderone exarkun@divmod.com:
On Wed, 31 Oct 2007 08:41:10 +0100, Nitro nitro@dr-code.org wrote:
Hello,
I wrote a pb application which also uses udp sockets. It works nicely. Now there's the situation where client and server are both located on the same machine. This means I could just connect to localhost and everything works. Unfortunately some firewalls warn about connections. I want to avoid this. So I thought about replacing the transports in pb with pipes instead of sockets if the server and client are both on the same machine. I'd have to do the same for my custom udp solution. Doing it for my custom udp code is easy, but how do I do this with pb? Do I just have to create some kind of reactor.createPipe() thing or is there more to it?
Since you're concerned about firewalls warning about connections, I'm guessing you're thinking about Windows. In that case, you can't use IReactorUNIX and IReactorUNIXDatagram, which would perhaps meet your use case (although for all I know, some firewall software warns about UNIX connections).
Yes, I am mainly working on windows.
There is no support for communicating over arbitrary pipes in Twisted. There is incidental support for communicating with pipes connected to another process created with reactor.spawnProcess, but the process API is not the same as the IProtocol API.
In general, using pipes will lessen performance anyway.
How big is the actual performance impact? The throughput right now is ~10-50 kbyte/s. Is there a better way than pipes? They were just the first thing that came to my mind. If pipes are still ok for my situation, how would I continue? Implement some kind of PipeCommunication class which implements IProtocol? What's the next step then? How do I plug this into pb?
Thanks for your time.
-Matthias
On Wed, 31 Oct 2007 14:24:50 +0100, Nitro nitro@dr-code.org wrote:
[snip]
How big is the actual performance impact? The throughput right now is ~10- 50 kbyte/s. Is there a better way than pipes? They were just the first thing that came to my mind.
TCP connections are the thing which is better. ;) Now that I think of it, though, I don't know anything about how pipes are implemented on Windows.
If pipes are still ok for my situation, how would I continue? Implement some kind of PipeCommunication class which implements IProtocol? What's the next step then? How do I plug this into pb?
You'll have to implement ITCPTransport based on a pipe. You can probably re-use some code in Twisted for this. For example, FileDescriptor from twisted.internet.abstract might provide part of the implementation for you.
You might want to look at a few other things in twisted.internet.interfaces, too (eg, IReactorFDSet).
Jean-Paul
Jean-Paul Calderone ha scritto:
On Wed, 31 Oct 2007 14:24:50 +0100, Nitro nitro@dr-code.org wrote:
[snip]
How big is the actual performance impact? The throughput right now is ~10- 50 kbyte/s. Is there a better way than pipes? They were just twaitformulipleobjecthe first thing that came to my mind.
TCP connections are the thing which is better. ;) Now that I think of it, though, I don't know anything about how pipes are implemented on Windows.
Windows named pipes should be the equivalent of Unix local sockets. They support overlapped (aka non blocking/asynchronous) access. http://msdn2.microsoft.com/en-us/library/aa365788.aspx and an example: http://msdn2.microsoft.com/en-us/library/aa365603.aspx
If I remember well, MySQL on Windows supports named pipes.
[...]
Manlio Perillo
Jean-Paul Calderone ha scritto:
On Wed, 31 Oct 2007 14:24:50 +0100, Nitro nitro@dr-code.org wrote:
[snip]
How big is the actual performance impact? The throughput right now is ~10- 50 kbyte/s. Is there a better way than pipes? They were just twaitformulipleobjecthe first thing that came to my mind.
TCP connections are the thing which is better. ;) Now that I think of it, though, I don't know anything about how pipes are implemented on Windows.
Windows named pipes should be the equivalent of Unix local sockets. They support overlapped (aka non blocking/asynchronous) access. http://msdn2.microsoft.com/en-us/library/aa365788.aspx and an example: http://msdn2.microsoft.com/en-us/library/aa365603.aspx
If I remember well, MySQL on Windows supports named pipes.
[...]
Manlio Perillo
The processworker module of my AsynQueue package spawns a process and makes a PB connection with it via stdin/stdout. Perhaps you could adapt it for your purposes.
http://foss.eepatents.com/trac/AsynQueue/browser/projects/AsynQueue/trunk/as...
Best regards, Ed
Nitro wrote:
Hello,
I wrote a pb application which also uses udp sockets. It works nicely. Now there's the situation where client and server are both located on the same machine. This means I could just connect to localhost and everything works. Unfortunately some firewalls warn about connections. I want to avoid this. So I thought about replacing the transports in pb with pipes instead of sockets if the server and client are both on the same machine. I'd have to do the same for my custom udp solution. Doing it for my custom udp code is easy, but how do I do this with pb? Do I just have to create some kind of reactor.createPipe() thing or is there more to it?
Thanks for your help, -Matthias
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Am 31.10.2007, 14:55 Uhr, schrieb Ed Suominen general@eepatents.com:
The processworker module of my AsynQueue package spawns a process and makes a PB connection with it via stdin/stdout. Perhaps you could adapt it for your purposes.
http://foss.eepatents.com/trac/AsynQueue/browser/projects/AsynQueue/trunk/as...
Best regards, Ed
Thanks, Ed!
This helped me to get started. I am using a modified version of StdIO along with _pollingfile now. It works really nicely!
-Matthias
participants (4)
-
Ed Suominen -
Jean-Paul Calderone -
Manlio Perillo -
Nitro