[Python-Dev] PEP 433: Choose the default value of the new cloexec parameter

[Cameron Simpson]

On 25Jan2013 12:24, Victor Stinner <victor.stinner at gmail.com> wrote:
| 2013/1/25 Charles-François Natali <cf.natali at gmail.com>:
| > You can actually count me in the cloexec=False camp, and against the
| > idea of a configurable default value.
| 
| Oh ok.

I'm leaning to this view myself also.

| > Why the default value shouldn't be tunable:
| > - I think it's useless: if the default cloexec behavior can be altered
| > (either by a command-line flag, an environment variable or a sys
| > module function), then libraries cannot rely on it and have to make
| > file descriptors cloexec on an individual basis, since the default
| > flag can be disabled.
| 
| In my experience, in most cases, the default value of cloexec just
| doesn't matter at all. If your program relies on the state of the
| close-on-exec flag: you have to make it explicit, specify cloexec
| paramater. Example: os.pipe(cloexec=True). If you don't modify your
| application, it will just not work using -e command line option (or
| PYTHONCLOEXEC environment variable). But why would you enable cloexec
| by default if your application is not compatible?

Because we're not all writing applications. I write a lot of small
library routines and then use them in my apps.

On a UNIX system I can plow on presuming cloexec=False, and will feel no
desire to label it in any calls. And so I might be making pipes or
whatever and expecting to pass them to subprocesses which will be told
the file descriptor numbers on the command line. (Let me say up front
that I haven't such an example in my code at present; generally that
kind of thing happens in shell script for me.)

Let someone change the global default and suddening my library code
doesn't work, and people will be tearing their hair out trying to figure
out why; because the code was written correctly in the old regime.

We have recently seen a mail thread where people were arguing (generally
correctly IMO) that programs should not do chdir, because it changes a
process wide global.

The situation with cloexec is in my mind very analogous.

So I am:
  +1 on offering a cloexec parameter, defaulting to False
  -0.5 on offering a global default that can be changed

[...]
| > - We should go over the standard library, and create FDs cloexec if
| > they're not handed over to the caller, either because they're
| > opened/closed before returning, or because the underlying file
| > descriptor is kept private (not fileno() method, although it's
| > relatively rare). That's the approach chosen by glibc, and it makes
| > sense: if another thread forks() while a thread is in the middle of
| > getpwnam(), you don't want to leak an open file descriptor to
| > /etc/passwd (or /etc/shadow).
| 
| I started to make cloexec explicit *everywhere* in the Python stdlib.
| I reverted my commit because I think that only a few application start
| child processes, so doing extra work (additionnal syscalls to set
| cloexec) would slowdown Python for no gain. See my revert commit to
| see how many functions need to be modified:
| http://hg.python.org/features/pep-433/rev/963e450fc24f

I see what you mean, it is a quite intrusive change.
However, what you're mostly backing out is cloexec=True.

There are therefore two changes coming here:

  - present the cloexec parameter

  - unilaterally make a global policy decision that in these many places
    in the stdlib, cloexec=True is now both correct/acceptable and
    _necessary_

If it were only the former, the change would be much smaller.

| That's why I really like the idea of being able to configure the
| default value of the cloexec parameter. By default: no overhead nor
| backward compatibility issue. Whereas you can set close-on-exec flag
| *everywhere* if you are concern by all issues of inheriting file
| descriptors (cases listed in the PEP).

Quietly breaking libraries that relied on cloexec=False being the status
quo...

The global tunable still make me feel uneasy.

I am in agreement that the general leakage of open files because cloexec
defaults to False is untidy and sometimes problematic; the tension here
isn't lost on me:-)

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Out on the road, feeling the breeze, passing the cars.  - Bob Seger