[Python-Dev] Proposal for a new function "open_noinherit" to avoid problems with subprocesses and security risks

[Stephen Hansen]

The kind of errors I mentioned ("permission denied" errors that
> seem to occur without an obvious reason) have cost me at least
> two weeks of debugging the hard way (with ProcessExplorer etc)
> and caused my manager to loose his trust in Python at all...
> I think it is well worth the effort to keep this trouble away from
> the Python programmers if possible.
>
> And throughout the standard library modules, "open" is used,
> causing these problems as soon as sub-processes come into play.
>
> Apart from shutil.copyfile, other examples of using open that can cause
> trouble are in socket.py (tell me any good reason why socket handles
> should be inherited to child processes) and even in logging.py.
>
> For example, I used RotatingFileHandler for logging my daemon
> program activity. Sometimes, the logging  itself caused errors,
> when a still-running child process had inherited the log file handle
> and log rotation occured.


I just wanted to express to the group at large that these experiences aren't
just Henning's; we spent a *tremendous* amount of time and effort debugging
serious problems that arose from file handles getting shared to subprocesses
where it wasn't really expected. Specifically, the RotatingFileHandler
example above. It blatantly just breaks when subprocesses are used and its
an extremely obtuse process to discover why.

It was very costly to the company because it came up at a bad time and was
*so* obtuse of an error. At first it looked like some sort of thread-safety
problem, so a lot of prying went into that before we got stumped... after
all, we *knew* no other process touched that file, and the logging module
(and RotatingFileHandler) claimed and looked thread-safe, so.. how could it
be having a Permission Denied error when it very clearly is closing the file
before rotating it? Eventually the culprit was found, but it was very
painful.

A couple similar issues have arisen since, and they're only slightly easier
to debug once you are expecting it. But the fact that the simple and obvious
features provided in the stdlib break as a result of you launching a
subprocess at some point sorta sucks :)

So, yeah. Anything even remotely or vaguely approaching Henning's patch
would be really, really appreciated.

--SH
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20070623/4c9eb540/attachment.htm