[Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets

[Victor Stinner]

2013/7/6 Charles-François Natali <cf.natali at gmail.com>:
>> I've read your "Rejected Alternatives" more closely and Ulrich
>> Drepper's article, though I think the article also supports adding
>> a blocking (default True) parameter to open() and os.open(). If you
>> try to change that default on a platform where it doesn't work, an
>> exception should be raised.
>
> Contrarily to close-on-exec, non-blocking only applies to a limited
> type of files (e.g. it doesn't work for regular files, which represent
> 90% of open() use cases).

What do you mean by "does not work"? On Linux, O_NONBLOCK flag can be
set on regular files, sockets, pipes, etc.

Example with a regular file on Linux 3.9:

smithers$ python3
Python 3.3.0 (default, Sep 29 2012, 22:07:38)
>>> from fcntl import *
>>> import os
>>> f=open("/etc/issue", "rb")
>>> fd=f.fileno()
>>> flags=fcntl(fd, F_GETFL)
>>> fcntl(fd, F_SETFL, flags|os.O_NONBLOCK)
0
>>> fcntl(fd, F_GETFL) & os.O_NONBLOCK
2048
>>> f.read(10)
b'Fedora rel'

For example, asyncore.file_dispatcher() uses fcntl to set the
O_NONBLOCK flag. (The asyncore module is probably the only module of
the stdlib which would benefit of a new os.set_blocking() function.)

> Also, one of the main reasons for exposing close-on-exec in
> open()/socket() etc is to make it possible to create file descriptors
> with the close-on-exec flag atomically, to prevent unwanted FD
> inheritance especially in multi-threaded code. And that's not
> necessary for the non-blocking parameter.

In the review of PEP 433 on this mailing list, Martin von Loewis said
that the PEP does guarantee the atomicity. The implementation of the
PEP falls back on ioctl or fcntl to set the flag on old Linux versions
(or if the OS does not support setting O_CLOEXEC flag when creating a
file descriptor). The GIL is released during system calls, so another
thread can call execv().

That's why I mentionned explicitly in the PEP 446 that it does not
offer any atomicity guarantee.

But open(filename, blocking=False) on Linux has at least one advantage
over f=open(filename); os.set_blocking(f.fileno()): it only uses one
syscall (versus 2 or 3 syscalls).

Victor