[Twisted-Python] Stdout from child of ProcessProtocol not propogating
We've been experimenting with creating an SSH virtual server for Windows based on conch. We've made good process, but we've run into a problem running child processes (children of spawnProcess()), in that stdout does not make it back to the client. I can demonstrate the problem using the example ptyserv.py script with a slight change for Windows. Here's the code to demonstrate the problem: from twisted.internet import reactor, protocol class FakeTelnet(protocol.Protocol): commandToRun = ['c:\\Windows\\System32\\cmd.exe'] def connectionMade(self): print 'connection made' self.propro = ProcessProtocol(self) reactor.spawnProcess(self.propro, self.commandToRun[0], self.commandToRun, os.environ) def dataReceived(self, data): self.propro.transport.write(data) def conectionLost(self, reason): print 'connection lost' self.propro.tranport.loseConnection() class ProcessProtocol(protocol.ProcessProtocol): def __init__(self, pr): self.pr = pr def outReceived(self, data): self.pr.transport.write(data) def processEnded(self, reason): print 'protocol connection lost' self.pr.transport.loseConnection() f = protocol.Factory() f.protocol = FakeTelnet reactor.listenTCP(5823, f) reactor.run() Run the above code on Windows, and telnet to localhost 5823. After you get the CMD.exe prompt, try to launch python. This is what you'll see: Microsoft Windows [Version 6.3.9600] (c) 2013 Microsoft Corporation. All rights reserved. C:\Temp>python python That's it. Hitting enter just drops down to another next blank line. Now what's interesting is that writes to stdin of the spawned CMD.exe get there. You can see this by pressing <CTRL>-D. The python subprocess will exit and you'll still be connected to your telnet session, now back at the CMD.exe prompt. I've experimented with modifying twisted.internet._dumbwin32proc.py so that the DuplicateHandle() calls (lines 147 - 163) are not called, thinking that inheritance was the cause, but no-go. I've also tried adding calls to msvcrt.set_mode() to os.O_BINARY to make sure windows isn't somehow cooking the child processes output, but that had no effect either. I've been banging my head against this for a week and I've reached a stuck point. I was wondering if someone could point me towards where I'm going wrong. Any advice would be appreciated.... FYI - once we get this worked out, we'd be happy to share the code. Thanks Jim C.
"Jim Carroll" <jim@carroll.com> writes:
After you get the CMD.exe prompt, try to launch python. This is what you'll
see:
Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.
C:\Temp>python
python
That's it. Hitting enter just drops down to another next blank line.
I suspect it's not a Twisted issue as much as a common risk running programs in a child process under Windows, especially executables using the standard C RTL with its default stdout buffering behavior. Do the built-in commands like DIR work ok? If so, then it's likely that it isn't that stdout isn't making it back to the client; rather it isn't even making it to your parent Twisted process. Usually it's a buffering or tty detection issue since a Windows child shell won't appear to be a tty. I'll note that python behaves the same way you describe when run from within a cmd shell under Cygwin's ssh server for example. In python's case, the C RTL is going into fully buffered mode. You should eventually get output once enough is generated (~8K) or the process exits. You can't control how the child executable will buffer from the parent unless the executable provides a way. So how you work around the issue depends on the actual command being run. Assuming this is the issue you're running into, using either "python -u" (if you just need output as it occurs) or "python -i" (for interactive use) should be a workaround. Or if you want to avoid clients having to remember the options, you can install appropriate environment variables (PYTHONUNBUFFERED or PYTHONINSPECT respectively) into the child process. But that's Python specific - there's no general solution for all executables, and there will be some for which you simply can't disable the buffering. The other thing you have to watch out for are executables you may run that actually use the Win32 console API rather than interact with stdin/stdout. Or certain methods of character inputs (like Python's msvcrt.getch). Again, not a Twisted issue, but you can't interact with that (unless you jump through some significant hoops). This shows up often with password prompts (which want to prevent echoing), curses like applications, or even in unexpected places for Y/N prompts in some utilities (like SC's default help, or NET TIME /SET's confirmation). Of course if you're just implementing the ssh server portion, you should be able to pretty much punt on it and leave it up to the user to determine how to best run whatever commands they are using. The behavior with your shell should be similar to other remote shell approaches. -- David
participants (2)
-
David Bolen -
Jim Carroll