2013/1/10 Charles-François Natali email@example.com:
Network servers like inetd or apache MPM (prefork) uses a process listening on a socket, and then fork to execute a request in a child process. I don't know how it works exactly, but I guess that the child process need a socket from the parent to send the answer to the client. If the socket is closed on execute (ex: Apache with CGI), it does not work :-)
Yes, but the above (setting close-on-exec by default) would *not* apply to stdin, stdout and stderr. inetd servers use dup(socket, 0); dup(socket, 1, dup(socket, 2) before forking, so it would still work.
Do you mean that file descriptors 0, 1 and 2 must be handled differently? A program can start without stdout (fd 1) nor stderr (fd 2), even if it's not the most common case :-)
For example, should "os.dup2(fd, 1)" set the close-on-exec flag on the file descriptor 1?
There are far more programs that are bitten by FD inheritance upon exec than programs relying on it, and whereas failures and security issues in the first category are hard to debug and unpredictable (especially in a multi-threaded program), a program relying on a FD that would be closed will fail immediately with EBADF, and so could be updated quickly and easily.
In other words, I think close-on-exec by default is probably a reasonable decision.
close-on-exec should probably have been the default in Unix, and is a much saner option.
I will update the PEP to at least mention this option (enable close-on-exec by default).
Note that if we do choose to set all file descriptors close-on-exec by default, there are several questions open:
- This would hold for open(), Socket() and other high-level
file-descriptor wrappers. Should it be enabled also for low-level syscall wrappers like os.open(), os.pipe(), etc?
os.pipe() has no higher level API, and it would be convinient to be able to set the close-on-exec flag, at least for the subprocess module
For os.open(), the question is more difficult. I proposed to add an cloexec keyword argument to os.open(). os.open(..., cloexec=False) would not do anything special, whereas os.open(..., cloexec=True) would use the best method to set the close-on-exec flag, including adding O_CLOEXEC or O_NOINHERIT to flags.
- On platforms that don't support atomic close-on-exec (e.g. open()
with O_CLOEXEC, socket() with SOCK_CLOEXEC, pipe2(), etc), this would require extra fcntl()/ioctl() syscalls. The cost is probably negligible, but we'd have to check the impact on some benchmarks.
Sure. We are working since a long time trying to reduce the number of syscalls at startup.
I proposed to use ioctl(fd, FIOCLEX) which only adds one syscall, instead of two for fcntl(fd, F_SETFD, new_flags) (old flags must be read to be future-proof, as adviced in the glibc documentation).
On Windows, Linux 2.6.23+ and FreeBSD 8.3+, no additional syscall is needed: O_NOINHERIT and O_CLOEXEC are enough (Linux just needs one additional syscall to ensure that O_CLOEXEC is supported).
So the worst case is when fcntl() is the only available option: 2 additional syscalls are required per creation of new file descriptors.
If setting close-on-exec has a significant cost, the configurable default value (sys.setdefaultcloexec()) should be considered more seriously.
While we are talking about enable close-on-exec by default and the number of syscalls: I would like to know how a new file descriptor can be created without close-on-exec flag. What would be the API? Most existing API are designed to *set* the flag, but how should define that the flag must be unset?
An option is to add an new "cloexec" keyword argument which would be True by default, and so set the argument to False to not set the flag. Example: os.pipe(cloexec=False). I dislike this option because developers may write os.pipe(cloexec=True) which would be useless and may be confusing if the flag is True by default.