[Twisted-Python] Perspective broker and pipes
![](https://secure.gravatar.com/avatar/2fe274d385cf2bb7f492d20a5631fc05.jpg?s=120&d=mm&r=g)
Hello, I wrote a pb application which also uses udp sockets. It works nicely. Now there's the situation where client and server are both located on the same machine. This means I could just connect to localhost and everything works. Unfortunately some firewalls warn about connections. I want to avoid this. So I thought about replacing the transports in pb with pipes instead of sockets if the server and client are both on the same machine. I'd have to do the same for my custom udp solution. Doing it for my custom udp code is easy, but how do I do this with pb? Do I just have to create some kind of reactor.createPipe() thing or is there more to it? Thanks for your help, -Matthias
![](https://secure.gravatar.com/avatar/7ed9784cbb1ba1ef75454034b3a8e6a1.jpg?s=120&d=mm&r=g)
On Wed, 31 Oct 2007 08:41:10 +0100, Nitro <nitro@dr-code.org> wrote:
Since you're concerned about firewalls warning about connections, I'm guessing you're thinking about Windows. In that case, you can't use IReactorUNIX and IReactorUNIXDatagram, which would perhaps meet your use case (although for all I know, some firewall software warns about UNIX connections). There is no support for communicating over arbitrary pipes in Twisted. There is incidental support for communicating with pipes connected to another process created with reactor.spawnProcess, but the process API is not the same as the IProtocol API. In general, using pipes will lessen performance anyway. Jean-Paul
![](https://secure.gravatar.com/avatar/2fe274d385cf2bb7f492d20a5631fc05.jpg?s=120&d=mm&r=g)
Am 31.10.2007, 12:59 Uhr, schrieb Jean-Paul Calderone <exarkun@divmod.com>:
Yes, I am mainly working on windows.
How big is the actual performance impact? The throughput right now is ~10-50 kbyte/s. Is there a better way than pipes? They were just the first thing that came to my mind. If pipes are still ok for my situation, how would I continue? Implement some kind of PipeCommunication class which implements IProtocol? What's the next step then? How do I plug this into pb? Thanks for your time. -Matthias
![](https://secure.gravatar.com/avatar/7ed9784cbb1ba1ef75454034b3a8e6a1.jpg?s=120&d=mm&r=g)
On Wed, 31 Oct 2007 14:24:50 +0100, Nitro <nitro@dr-code.org> wrote:
TCP connections are the thing which is better. ;) Now that I think of it, though, I don't know anything about how pipes are implemented on Windows.
You'll have to implement ITCPTransport based on a pipe. You can probably re-use some code in Twisted for this. For example, FileDescriptor from twisted.internet.abstract might provide part of the implementation for you. You might want to look at a few other things in twisted.internet.interfaces, too (eg, IReactorFDSet). Jean-Paul
![](https://secure.gravatar.com/avatar/4e1ae4b836a9cfe3945d8c661b37246b.jpg?s=120&d=mm&r=g)
Jean-Paul Calderone ha scritto:
Windows named pipes should be the equivalent of Unix local sockets. They support overlapped (aka non blocking/asynchronous) access. http://msdn2.microsoft.com/en-us/library/aa365788.aspx and an example: http://msdn2.microsoft.com/en-us/library/aa365603.aspx If I remember well, MySQL on Windows supports named pipes.
[...]
Manlio Perillo
![](https://secure.gravatar.com/avatar/4e1ae4b836a9cfe3945d8c661b37246b.jpg?s=120&d=mm&r=g)
Jean-Paul Calderone ha scritto:
Windows named pipes should be the equivalent of Unix local sockets. They support overlapped (aka non blocking/asynchronous) access. http://msdn2.microsoft.com/en-us/library/aa365788.aspx and an example: http://msdn2.microsoft.com/en-us/library/aa365603.aspx If I remember well, MySQL on Windows supports named pipes.
[...]
Manlio Perillo
![](https://secure.gravatar.com/avatar/7905d47e9096477130b7055ab70d0f30.jpg?s=120&d=mm&r=g)
The processworker module of my AsynQueue package spawns a process and makes a PB connection with it via stdin/stdout. Perhaps you could adapt it for your purposes. http://foss.eepatents.com/trac/AsynQueue/browser/projects/AsynQueue/trunk/as... Best regards, Ed Nitro wrote:
![](https://secure.gravatar.com/avatar/7ed9784cbb1ba1ef75454034b3a8e6a1.jpg?s=120&d=mm&r=g)
On Wed, 31 Oct 2007 08:41:10 +0100, Nitro <nitro@dr-code.org> wrote:
Since you're concerned about firewalls warning about connections, I'm guessing you're thinking about Windows. In that case, you can't use IReactorUNIX and IReactorUNIXDatagram, which would perhaps meet your use case (although for all I know, some firewall software warns about UNIX connections). There is no support for communicating over arbitrary pipes in Twisted. There is incidental support for communicating with pipes connected to another process created with reactor.spawnProcess, but the process API is not the same as the IProtocol API. In general, using pipes will lessen performance anyway. Jean-Paul
![](https://secure.gravatar.com/avatar/2fe274d385cf2bb7f492d20a5631fc05.jpg?s=120&d=mm&r=g)
Am 31.10.2007, 12:59 Uhr, schrieb Jean-Paul Calderone <exarkun@divmod.com>:
Yes, I am mainly working on windows.
How big is the actual performance impact? The throughput right now is ~10-50 kbyte/s. Is there a better way than pipes? They were just the first thing that came to my mind. If pipes are still ok for my situation, how would I continue? Implement some kind of PipeCommunication class which implements IProtocol? What's the next step then? How do I plug this into pb? Thanks for your time. -Matthias
![](https://secure.gravatar.com/avatar/7ed9784cbb1ba1ef75454034b3a8e6a1.jpg?s=120&d=mm&r=g)
On Wed, 31 Oct 2007 14:24:50 +0100, Nitro <nitro@dr-code.org> wrote:
TCP connections are the thing which is better. ;) Now that I think of it, though, I don't know anything about how pipes are implemented on Windows.
You'll have to implement ITCPTransport based on a pipe. You can probably re-use some code in Twisted for this. For example, FileDescriptor from twisted.internet.abstract might provide part of the implementation for you. You might want to look at a few other things in twisted.internet.interfaces, too (eg, IReactorFDSet). Jean-Paul
![](https://secure.gravatar.com/avatar/4e1ae4b836a9cfe3945d8c661b37246b.jpg?s=120&d=mm&r=g)
Jean-Paul Calderone ha scritto:
Windows named pipes should be the equivalent of Unix local sockets. They support overlapped (aka non blocking/asynchronous) access. http://msdn2.microsoft.com/en-us/library/aa365788.aspx and an example: http://msdn2.microsoft.com/en-us/library/aa365603.aspx If I remember well, MySQL on Windows supports named pipes.
[...]
Manlio Perillo
![](https://secure.gravatar.com/avatar/4e1ae4b836a9cfe3945d8c661b37246b.jpg?s=120&d=mm&r=g)
Jean-Paul Calderone ha scritto:
Windows named pipes should be the equivalent of Unix local sockets. They support overlapped (aka non blocking/asynchronous) access. http://msdn2.microsoft.com/en-us/library/aa365788.aspx and an example: http://msdn2.microsoft.com/en-us/library/aa365603.aspx If I remember well, MySQL on Windows supports named pipes.
[...]
Manlio Perillo
![](https://secure.gravatar.com/avatar/7905d47e9096477130b7055ab70d0f30.jpg?s=120&d=mm&r=g)
The processworker module of my AsynQueue package spawns a process and makes a PB connection with it via stdin/stdout. Perhaps you could adapt it for your purposes. http://foss.eepatents.com/trac/AsynQueue/browser/projects/AsynQueue/trunk/as... Best regards, Ed Nitro wrote:
participants (4)
-
Ed Suominen
-
Jean-Paul Calderone
-
Manlio Perillo
-
Nitro