do anonymous pipes normally work on NT?

[Bill Tutt]

> From: Georg Mischler [mailto:schorsch at schorsch.com]

> The Documentaion provided with VC++ about this stuff
> it not very helpful, so you have to try and guess.
> Here's what I figured by now: The CreateProcess()
> function has several ways to pass handles to child
> processes. The way I tried it (passing through
> StartupInfo.hStdXxx) only seems to work with disk files,
> and not with pipes.
> 

It may seem that way, but thats far from the truth.
Because win32popen.cpp uses StartupInfo.hStdxxxx and it works quite well.
The real cause of this problem is that there is a bug in win32process's
startupinfo support. (see info at bottom of email) 

> The other way is to assign the handles to the stdxxx
> of the current process, and inherit them to the
> child. This does work with pipes, and is fundamentally
> incompatible with the first approach. In fact, as soon
> as you just tell it you want to pass any handles in the
> StartupInfo, by assigning STARTF_USESTDHANDLES to it's
> dwFlags attribute, inheritance is dead. Still, if
> you *want* to pass (file) handles through the
> StartupInfo object, you nonetheless *need* to set the
> bInheritHandles parameter of CreateProcess() to TRUE.
> Go figure...
> 

Well, inheritance of stdin, stdout, stderr is dead, yes. (All other
inhertible handles will be valid in the child process)
They sure like to make life confusing don't they? :)

> The least obvious thing though, which I only found
> by comparing the working code with mine very carefully,
> is that for inheritance to work, the dwCreationFlags
> must *not* contain win32process.DETACHED_PROCESS.
> The VisualStudio docs (and consequently the win32proc
> help file) tell us that this would deny acess to the
> *console* of the parent process. In fact, it appears
> to cut all connections between the two processes.
> 

In theory, this should only disallow the new process from acessing the
current program's NT Console (if any). This should have nothing to do with
hStdXxx being anonymous pipe handles. Although after some experimentation it
certainly seems to be doing just a little bit more than revoking access to
the NT console. i.e. passing DETACHED_PROCESS in the example below works
fine, but it completly failed if I tried to fire up Mark's cool debugger.
(It asserted at objcore.cpp:45 in MFC land...)

> In my latest experiments this *does* work if the parent
> process manages to get the handles across. CreateProcess()
> has so many mutually exclusive combinations of parameters
> that it's just hard to find the right ones for a particular
> purpose.
> 

Hrm, now when I go to retest what I did last night, I wasn't paying close
enough attention.  If i do "cat.py < xxx > yyy" it complains because
something in the file extension association code is screwing things up. :(

python cat.py < xxxx > yyy works just fine.

> I don't mind setting and resetting the std handles in the
> parent, as long as I get the result I want. But then, I'm
> also trying to get along with as little knowledge as possible
> about Windows programming. I assume you look at it from a
> slightly different angle.
> 

Heh, if you know more about it you can do more useful things when life
becomes complicated. :) I'm just digging through the docs like you, I don't
work on the NT team and can't go look up the source for CreateProcess(). 

Attached below is the fix information, along with a Georg's example slightly
tweaked to be useful.

Bill

Patch info:

Alter the PyHANDLE_Check() branch of win32processmodule_win32.cpp:sethandle
to:
		*pobHandle = v;
		if (PyWinObject_AsHANDLE(v, ph))
		{
			Py_INCREF(v);
		}
		else
		{
			rc = -1;
		}
The cast that was there before was setting *ph equal to the Python reference
count on v.

Working sample:

'''cat.py
a "enhanced" version of the good old cat unix filter.
'''

import sys
try:
    # write status to a file, in order not to lose it.
    f = open('log.txt', 'a')
    f.write('before write\n')
    sys.stdout.write('huh?')
    f.write('after write, before read\n')
    data = sys.stdin.read()
    f.write('data: ' + data)
    f.write('after read\n')
    f.write('in loop, before write\n')
    sys.stdout.write(data)
except:
    import traceback, string
    tb_strs = traceback.format_exception(
            sys.exc_type,
            sys.exc_value,
            sys.exc_traceback)
    f.write(string.join(tb_strs))

# end of cat.py

'''runproc.py

start a process with three inherited pipes.
Try to write to and read from those.
'''

import win32pipe
import win32file
import win32process
import win32security
import win32con

class Process:
    def run(self, cmdline):
        # security attributes for pipes
        sAttrs = win32security.SECURITY_ATTRIBUTES()
        sAttrs.bInheritHandle = 1

        # create pipes
        hStdin_r,  self.hStdin_w  = win32pipe.CreatePipe(sAttrs, 0)
        self.hStdout_r, hStdout_w = win32pipe.CreatePipe(sAttrs, 0)
        self.hStderr_r, hStderr_w = win32pipe.CreatePipe(sAttrs, 0)

        # set the info structure for the new process.
        StartupInfo = win32process.STARTUPINFO()
        StartupInfo.hStdInput  = hStdin_r
        StartupInfo.hStdOutput = hStdout_w
        StartupInfo.hStdError  = hStderr_w
        StartupInfo.dwFlags = win32process.STARTF_USESTDHANDLES 

        # start the process.
        hProcess, hThread, dwPid, dwTid = win32process.CreateProcess(
                None,   # program
                cmdline,# command line
                None,   # process security attributes
                None,   # thread attributes
                1,      # inherit handles, or USESTDHANDLES won't work.
                        # creation flags. Don't access the console.
                win32process.DETACHED_PROCESS,
                None,   # no new environment
                None,   # current directory (stay where we are)
                StartupInfo)
        # normally, we would save the pid etc. here...

        err, bytes = win32file.WriteFile(self.hStdin_w, 'hmmm\n')
        print 'wrote:', bytes, 'bytes'

        res, str = win32file.ReadFile(self.hStdout_r, 7)
        if res == 0: print 'read:', str
        else: print 'read nothing on stdin.'

if __name__ == '__main__':
    p = Process()
    p.run(r'd:\pythondevtree\python\dist\src\pcbuild\python_d.exe
d:\cat.py')

# end of runproc.py