[ python-Bugs-566037 ] Popen exectuion blocking w/threads
SourceForge.net
noreply at sourceforge.net
Tue Mar 22 23:11:29 CET 2005
Bugs item #566037, was opened at 2002-06-07 21:07
Message generated for change (Comment added) made by facundobatista
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=566037&group_id=5470
Category: Threads
Group: Python 2.2.1
>Status: Closed
>Resolution: Wont Fix
Priority: 1
Submitted By: Martin Stoufer (mstoufer)
Assigned to: Nobody/Anonymous (nobody)
Summary: Popen exectuion blocking w/threads
Initial Comment:
The following unit test hangs after the first two lines
of output. This is wholly reproducible (for us) under
2.2.1 and totaly unreproducible under 2.1. We have both
interpreters installed on a RH7.2 PC with linkings
against the apparent same /lib/libthread.so.0
import os, threading,time, sys
def read_output(self, cmd, timeout):
print "run: %s" % cmd
(inf,outf) = os.popen4(cmd)
print "started! out_fd=%d" % outf.fileno()
while 1:
line = outf.readline()
if line == "": break
print len(line),line
sys.stdout.flush()
return
if __name__ == '__main__':
thr =
threading.Thread(target=read_output,args=(1,"ping -c 5
www.mit.edu",0))
thr.start()
print "Started thread"
while 1: time.sleep(1)
mstoufer at dpsslx05(3)>ldd /usr/local/bin/python2.{1,2}
/usr/local/bin/python2.1:
libpthread.so.0 => /lib/libpthread.so.0 (0x40029000)
libdl.so.2 => /lib/libdl.so.2 (0x40040000)
libutil.so.1 => /lib/libutil.so.1 (0x40044000)
libstdc++-libc6.1-2.so.3 =>
/usr/local/lib/libstdc++-libc6.1-2.so.3 (0x40047000)
libm.so.6 => /lib/libm.so.6 (0x4008f000)
libc.so.6 => /lib/libc.so.6 (0x400b1000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
/usr/local/bin/python2.2:
libdl.so.2 => /lib/libdl.so.2 (0x40029000)
libpthread.so.0 => /lib/libpthread.so.0 (0x4002d000)
libutil.so.1 => /lib/libutil.so.1 (0x40044000)
libm.so.6 => /lib/libm.so.6 (0x40047000)
libc.so.6 => /lib/libc.so.6 (0x4006a000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
----------------------------------------------------------------------
Comment By: Facundo Batista (facundobatista)
Date: 2005-03-22 19:11
Message:
Logged In: YES
user_id=752496
Deprecated. Reopen only if still happens in 2.3 or newer.
. Facundo
----------------------------------------------------------------------
Comment By: Martin Stoufer (mstoufer)
Date: 2004-11-30 12:18
Message:
Logged In: YES
user_id=133905
I have not worked on this bug since I submitted it and have
since gone onto a more saner software model. This bug may be
closed or assigned to someone else to follow up on.
----------------------------------------------------------------------
Comment By: Facundo Batista (facundobatista)
Date: 2004-11-29 20:28
Message:
Logged In: YES
user_id=752496
Works fine in Py2.3.3, Fedora Core 2.
----------------------------------------------------------------------
Comment By: Facundo Batista (facundobatista)
Date: 2004-11-29 20:28
Message:
Logged In: YES
user_id=752496
Please, could you verify if this problem persists in Python 2.3.4
or 2.4?
If yes, in which version? Can you provide a test case?
If the problem is solved, from which version?
Note that if you fail to answer in one month, I'll close this bug
as "Won't fix".
Thank you!
. Facundo
----------------------------------------------------------------------
Comment By: Becky Burwell (burwell)
Date: 2002-10-04 19:01
Message:
Logged In: YES
user_id=348250
FYI: this happens if the program is a bash shell which calls
ping.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2002-10-04 18:20
Message:
Logged In: YES
user_id=6380
I can reproduce this in Python 2.3, by the way. I'm just not
sure that there's anything that Python can do to avoid this
-- it feels like a bug in libc (or in ping).
Lowering priority because of that.
----------------------------------------------------------------------
Comment By: Becky Burwell (burwell)
Date: 2002-10-04 17:53
Message:
Logged In: YES
user_id=348250
We have the same problem in RedHat 7.3 with Python 2.2
using ping in a thread. If I do a popen or just os.system in a
thread, the ping hangs for hosts that are not responding
(works fine IF the host is up.)
Note: this code works fine in Python 2.1.1 and works fine not
in a thread in Python 2.2
Also: if the command prints out lots of output there is still a
hang.
The ping command has to be killed with a kill -9.
#!/usr/bin/python2
import os,threading
class MyHost(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
ping_string = "/bin/ping -q -c 1 -w 2 hostthatisdown"
print "about to execute ping string %s\n" % ping_string
f = os.popen (ping_string,"r")
result = f.close ()
print "result is ",result
thread = MyHost()
thread.start ()
----------------------------------------------------------------------
Comment By: Martin Stoufer (mcstoufer)
Date: 2002-07-24 13:31
Message:
Logged In: YES
user_id=243169
I've found that using the generic os.popen() call and just
reading everything back to be a somewhat equitable fix. The
returned object is file like and the .close() call on it at
the end seems to clean up house pretty well. Before, with a
Popen4() object, I was getting a lot of <defunct> processes.
Now they all go away nicely. I'd be willing to follow up in
person with anyone here, MCStoufer at lbl.gov
----------------------------------------------------------------------
Comment By: Zoran Bosnjak (zoranbosnjak)
Date: 2002-07-24 10:28
Message:
Logged In: YES
user_id=583469
I have the same problem with python2.2 (the child
process does not terminate on SIGTERM - it only
terminates on SIGKILL). The same program works
(terminates) fine with python2.1.1 (all other libs are the
same). Here is my simple testfile:
#!/usr/bin/env python2.2
import os, time, signal, threading, string
from popen2 import Popen3, Popen4
pid = 0
def run():
global pid
f = Popen4('ping localhost')
pid = f.pid
while 1:
s = f.fromchild.readline()
if not s: break
t = threading.Thread(target=run)
t.start()
time.sleep(10)
print 'timeout'
os.kill(pid, signal.SIGTERM)
t.join()
print 'bye'
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2002-06-12 01:16
Message:
Logged In: YES
user_id=6380
Anything is possible. I suggest asking for help in a Linux
threading guru forum.
----------------------------------------------------------------------
Comment By: Martin Stoufer (mcstoufer)
Date: 2002-06-12 01:10
Message:
Logged In: YES
user_id=243169
This same issue arises when any subprocess is declared that makes
more than 1 network read/write. From the system side, the Ping process
is simply sleeping; we feel that the process is looking for the pipe to
empty so it can finish it's write to stdout.
Replacing cmd with 'ping -c 1 www.mit.edu' works everytime.
Replacing cmd with 'echo\'Hello. world\'' works as well.
We have beeen using a therading model similar to this under 2.1 and it
has been quite robust in its execution. Is it possible that this is a
compile-time issue with the thread linking?
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2002-06-10 16:39
Message:
Logged In: YES
user_id=6380
Mixing forks and threads is often asking for trouble...
Does this also hang with another command that produces more
than two lines of output, or is it limited to ping?
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2002-06-09 05:32
Message:
Logged In: YES
user_id=21627
I can't reproduce this on a SuSE system, either (which has
glibc 2.2.5), so I'm tempted to declare it a glibc 2.1 bug.
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-06-08 02:42
Message:
Logged In: YES
user_id=14198
Just noting that current CVS Python works fine on Windows,
but also hangs on RH7. My Linux debuggings skills are such
that I can offer no further help ;)
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=566037&group_id=5470
More information about the Python-bugs-list
mailing list