[Python-Dev] Add "e" (close and exec) mode to open()

[Victor Stinner]

Oops, I sent my email too early by mistake (it was not finished).

> The myriad cloexec
> APIs between different platforms suggests to me that using this
> features requires understanding its various quirks on different
> platforms.

Sorry, I don't understand. What do you mean by "various quirks". The
"close-on-exec" feature is implemented differently depending on the
platform, but it always have the same meaning. It closes the file when
a subprocess is created. Running a subprocess is also implemented
differently depending on the OS, there are two mains approaches:
fork()+exec() on UNIX, <something else> on Windows (I don't know how
it works on Windows).

Extract of fcntl() manual page on Linux: "If the FD_CLOEXEC  bit  is
0, the file descriptor will remain open across an execve(2), otherwise
it will be closed."

I would like to expose the OS feature using a portable API to hide the
"The myriad cloexec APIs".

Victor

2013/1/8 Victor Stinner <victor.stinner at gmail.com>:
> Benjamin wrote:
>> I'm not sure it's worth cluttering the open() interface with such a
>> non-portable option.
> Zbyszek wrote:
>> If the best-effort fallback is included, it is quite portable. Definitely
>> all modern and semi-modern systems support either the atomic or the
>> nonatomic methods.
> Gregory wrote:
>> I'm not excited about raising an exception when it isn't supported;
>> it should attempt to get the same behavior via multiple API calls instead
>
> Yes, I'm proposing the best-effort approach: use O_CLOEXEC/O_NOINHERIT
> if available, or fcntl()+FD_CLOEXEC otherwise.
>
> My patch requires fcntl() + FD_CLOEXEC flag or open() + O_NOINHERIT
> flag. I failed to find an OS where none of these flag/function is
> present.
>
> Usually, when I would like to test the portability, I test Linux,
> Windows and Mac OS X. Then I test FreeBSD, OpenBSD and OpenIndiana
> (Solaris). I don't test other OS because I don't know them and they
> are not installed on my PC :-) (I have many VM.)
>
> Should I try other platforms?
>
> Benjamin wrote:
>> People requiring such control should use the low-level os.open interface.
>
> os.open() is not very convinient because you have to chose the flag
> and the functions depending on the OS. If the open() flag is rejected,
> we should at least provide an helper for os.open() + fcntl().
>
> The myriad cloexec
> APIs between different platforms suggests to me that using this
> features requires understanding its various quirks on different
> platforms.
>
> Victor
>
> 2013/1/8 Benjamin Peterson <benjamin at python.org>:
>> 2013/1/7 Gregory P. Smith <greg at krypto.org>:
>>>
>>>
>>>
>>> On Mon, Jan 7, 2013 at 4:03 PM, Benjamin Peterson <benjamin at python.org>
>>> wrote:
>>>>
>>>> 2013/1/7 Victor Stinner <victor.stinner at gmail.com>:
>>>> > Hi,
>>>> >
>>>> > I would like add a new flag to open() mode to close the file on exec:
>>>> > "e". This feature exists using different APIs depending on the OS and
>>>> > OS version: O_CLOEXEC, FD_CLOEXEC and O_NOINHERIT. Do you consider
>>>> > that such flag would be interesting?
>>>>
>>>> I'm not sure it's worth cluttering the open() interface with such a
>>>> non-portable option. People requiring such control should use the
>>>> low-level os.open interface.
>>>
>>>
>>> The ability to supply such flags really belongs on _all_ high or low level
>>> file descriptor creating APIs so that things like subprocess_cloexec_pipe()
>>> would not be necessary:
>>> http://hg.python.org/cpython/file/0afa7b323abb/Modules/_posixsubprocess.c#l729
>>
>> I think the open() interface should have consistent and
>> non-conditional support features to the maximum extent possible. The
>> recent addition of "x" is a good example I think. The myriad cloexec
>> APIs between different platforms suggests to me that using this
>> features requires understanding its various quirks on different
>> platforms.
>>
>>
>>
>> --
>> Regards,
>> Benjamin