[Python-Dev] Proposal for a new function "open_noinherit" to avoid problems with subprocesses and security risks

["Martin v. Löwis"]

>> Putting it into the library is fine. However, we need to find
>> an implementation strategy that meets the user's needs, and
>> is still maintainable.
>>
>> Python 3 will offer a clean solution, deviating entirely from
>> stdio.
> 
> Let me point out that stdio is not the problem.
> The problem is handle inheritance.
> So I don't see how this is going to be solve in Python 3 just by
> not using stdio.

In Python 3, it would be possible to implement the "n" flag
for open(), as we call CreateFile directly.

> And to open a file non-inheritable should be possible in an easy
> and platform-independent way for the average python programmer.

I don't see why it is a requirement to *open* the file in
non-inheritable mode. Why is not sufficient to *modify*
an open file to have its handle non-inheritable in
an easy and platform-independent way?

> Maybe this is why my HTTP Server sometimes seems to not
> react when a subprocess is running...
> If more than one process has a handle for the same socket,
> how does the OS know which process should react?

The processes which don't perform accept(), recv(), or
select() operations are not considered by the operating
system. So if only one process does recv() (say), then
this process will read the data.

If multiple processes perform accept() (which is a common
case), the system selects a process at random. This is
desirable, as the system will then automatically split
the load across processes, and the listen backlog cannot
pile up: if multiple connection requests arrive at the
same time, one process will do accept, and then start
to process the connection. Then the second process will
take the second request, and so on.

If multiple processes perform recv(), the system will
again chose randomly. This is mostly undesirable, and
should be avoided.

>> With that API, it would be possible to provide cross-platform
>> access to the close-on-exec flag. Applications interested in setting
>> it could then set it right after opening the file.
> 
> YES - that's exactly why I proposed an open_noinherit function.

I think I missed that proposal. What would that function do?

If you propose it to be similar to the open() function, I'd
be skeptical. It's not possible to implement that in thread-safe
way if you use SetHandleInformation/ioctl.

Regards,
Martin