subprocess.Popen inheriting

[Gabriel Genellina]

En Thu, 18 Dec 2008 19:46:45 -0200, Aaron Brady <castironpi at gmail.com>  
escribió:

> On Dec 17, 7:16 pm, "Gabriel Genellina" <gagsl-... at yahoo.com.ar>
> wrote:
>> En Wed, 17 Dec 2008 22:46:32 -0200, Aaron Brady <castiro... at gmail.com>  
>> escribió:
>>
>>
>>
>> > On Dec 17, 5:05 pm, "Gabriel Genellina" <gagsl-... at yahoo.com.ar>
>> > wrote:
>> >> En Wed, 17 Dec 2008 12:21:38 -0200, Jeremy Sanders  
>> >> <jeremy+complangpyt... at jeremysanders.net> escribió:
>>
>> >> > It would be nice if Python created pipes that are properly  
>> >> inheritable by
>> >> > default by child processes, as they're mostly used for IPC.
>>
>> >> I'd say it is a bug in os.pipe implementation; they should be  
>> >> inheritable  
>> >> by default, as in posix (after all, the code is in "posixmodule.c").
>>
>> > The code looks like this:
>>
>> >    ok = CreatePipe(&read, &write, NULL, 0);
>> >    Py_END_ALLOW_THREADS
>> >    if (!ok)
>> >            return win32_error("CreatePipe", NULL);
>> >    read_fd = _open_osfhandle((Py_intptr_t)read, 0);
>> >    write_fd = _open_osfhandle((Py_intptr_t)write, 1);
>>
>> > 'If lpPipeAttributes is NULL, the handle cannot be inherited.'  You
>> > could populate a 'SECURITY_ATTRIBUTES' structure, or call
>> > DuplicateHandle on both of them.
>>
>> > A patch would look like this:
>>
>> > SECURITY_ATTRIBUTES sattribs;
>> > sattribs.nLength = sizeof(sattribs);
>> > sattribs.lpSecurityDescriptor = NULL;
>> > sattribs.bInheritHandle = TRUE;
>> > ok = CreatePipe(&read, &write, &sattribs, 0);
>>
>> Yes, that's exactly how os.popen does it (in posixmodule.c)
>>
>> > This still doesn't answer whether the file descriptor return by
>> > '_open_osfhandle' can be inherited too.
>>
>> It doesn't matter. The OS only cares about file handles, not C RTL  
>> structures.
>
> Sorry for the multiple posts.  File handles are inheritable by child
> processes, if the permissions are right.  File descriptors are not.
> Is there a way that we can get the handles of a pipe into code, so
> that we can pass them to a subprocess?

On Windows, file handles are the real OS stuff, the "true" reference to an  
open file. File descriptors are not, they exist only to please the C  
runtime library. Programs not written in C (directly, or indirectly like  
Python) don't care at all about file descriptors. And in case one actually  
cares, there is _open_osfhandle in the C RTL (available as  
msvcrt.open_osfhandle from Python).
A subprocess may inherit handles from its parent [there are two filters:  
the parameter "bInheritHandles" in the CreateProcess call provides global  
control, and individual handles can be made inheritable or not, before  
creating the new subprocess].
"Anonymous" pipes are good to replace stdin/stdout/stderr, because there  
is no need to explicitely communicate the handle value to the subprocess:  
one just replaces the corresponding handle with the desired pipe, and the  
subprocess might not even notice it.
In case this is not enough, one might pass the handle (as a number) in the  
command line, but probably a "named pipe" would be better. As this is not  
transparent for the child process, one must explicitely code such things.

> Will it take calling
> 'CreatePipe' from ctypes directly if on Windows?  Or can 'os.pipe' be
> made to abstract that?  If Windows can't inherit descriptors,
> 'os.pipe' should return handles, and 'os.read' &co. should accept
> them.

I think the best way would be to modify os.pipe so it returns inheritable  
pipes, as it should have been from the beginning.

> It is a fairly large patch.

Not at all, you have already posted most of it.

-- 
Gabriel Genellina