[New-bugs-announce] [issue23213] subprocess communicate() hangs when stderr isn't closed

[Thomas D.]

New submission from Thomas D.:

Hi,

to demonstrate the problem you need >=systemd-217:

# python3.4
Python 3.4.2 (default, Oct 12 2014, 20:09:43)
[GCC 4.8.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> sp = subprocess.Popen(["/sbin/udevd", "--daemon"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> out, err = sp.communicate()

[hangs]

"ps" will show

root       9619  0.0  0.1  23340  5404 pts/5    Ss   Jan09   0:00  \_ -bash
root      13291  0.0  0.2  45352  9784 pts/5    S+   00:34   0:00      \_ python3.4
root      13311  0.0  0.0      0     0 pts/5    Z+   00:34   0:00          \_ [udevd] <defunct>

Calling "/sbin/udevd --daemon" from the shell works fine.

>>> errorlog = open("/tmp/stderr.log", "wb")
>>> sp = subprocess.Popen(["/sbin/udevd", "--daemon"], stdout=subprocess.PIPE, stderr=errorlog)

works, too.

The problem first appeared in systemd-217. I bisected systemd's source code and the commit since when Python's subprocess module is unable to start udevd is

https://github.com/systemd/systemd/commit/5c67cf2774a8b964f4d7cd92a4c447da81ac6087


This is not a systemd/udev only problem. The problem was first seen with the php-fpm daemon from PHP (but only when using "error_log = syslog").

Please see the original bug report at https://github.com/saltstack/salt/issues/14957 for more details.

Because Salt is still at Python 2.7, the problem can be reproduced with Python 2.7, too.


Is it a bug in subprocess? In systemd/PHP? Are we (salt) using subprocess the wrong way?

Thanks!


PS:
On your system, "/sbin/udevd" will be probably "/lib/systemd/systemd-udevd"

Not sure if this is related to http://bugs.python.org/issue12786 in some ways.

----------
components: Library (Lib)
messages: 233788
nosy: whissi
priority: normal
severity: normal
status: open
title: subprocess communicate() hangs when stderr isn't closed
type: behavior
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23213>
_______________________________________