[issue11284] slow close file descriptors in subprocess, popen2, os.popen*

Gregory P. Smith report at bugs.python.org
Wed Mar 2 20:11:32 CET 2011


Gregory P. Smith <greg at krypto.org> added the comment:

There is no OS API call to provide a *safe* way to get a list of all open file descriptors as part of POSIX in general that can be called after the fork() and before the exec().  It must be async signal safe. The closefrom() call available in Solaris and recent BSDs looks useful, is it async signal safe?  I still want to find a way to do this nicely on Linux (even if it means me going and implementing a closefrom syscall to be added to 2.6.39).

Your posix_closefrom() implementation as written today is not safe to call between fork() and exec() due to the opendir/readdir implementation.  It can and will hang processes at unexpected times.


Another thought that I've seen done in other subprocess implementations as a compromise:

Offer a way to use the hacky *not guaranteed to close everything if the process is multithreaded* version that has the parent get the list of fds before the fork() and the child closing those before exec().  That leaves the race condition of new fds being opened between the creation of that list and the fork() but would be fast.  If we can detect if any other threads exist in the program (they may have been created by C extension modules or by a C program that is embedding Python within it) we could conditionally use that approach vs the close-everything-possible approach so that people using this with generic programs that don't involve threads are not so heavily impacted.

----------
assignee:  -> gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11284>
_______________________________________


More information about the Python-bugs-list mailing list