[Python-Dev] Proposal for a new function "open_noinherit" to avoid problems with subprocesses and security risks

["Martin v. Löwis"]

>> I don't see why it is a requirement to *open* the file in
>> non-inheritable mode. Why is not sufficient to *modify*
>> an open file to have its handle non-inheritable in
>> an easy and platform-independent way?
> 
> Threads. Consider that you may fork a process on one thread right
> between the calls to open() and fcntl(F_SETFD, FD_CLOEXEC) on another
> thread. The only way to be safe is to open the file non-inheritable to
> start with.

No, that's not the only safe way. The application can synchronize the
threads, and prevent starting subprocesses during critical regions.
Just define a subprocess_lock, and make all of your threads follow
the protocol of locking that lock when either opening a new file,
or creating a subprocess.

> Now, it is currently impossible under linux to open a file descriptor
> noninheritable, but they're considering adding that feature (I don't
> have the thread-refs on me, but it's actually from the last month). The
> issue is that there's a *bunch* of syscalls that open FDs: this feature
> would need to be added to all of them, not only "open".

Right. That is what makes it difficult inherently on the API level.

> It's possible that it makes sense for python to provide "as good as
> possible" an implementation. At the least, putting the fcntl call in the
> same C function as open would fix programs that don't open files/spawn
> processes outside of the GIL protection.

No, that would not work. Python releases the GIL when opening a file
(and needs to do so because that may be a long-running operation).

> Now I'm confused: are you talking about the same thread-safety situation
> as I described above? 

Yes.

> If so, why did you ask why it's not sufficient to
> modify a handle to be non-inheritable?

Because I wanted to hear what the reasons are to consider that
insufficient. I would have expected "ease of use" and such things
(perhaps Henning will still bring up other reasons). If
thread-safety is a primary concern, then that flag should *not*
be added to open(), since it cannot be implemented in a
thread-safe manner in a generic way - only the application can
perform the proper synchronization. As discussed, there are
other handles subject to inheritance, too, and the application
would have to use the modify-handle function, anyway, which
means it needs to make it thread-safe through explicit locking.

Regards,
Martin