[Python-bugs-list] [ python-Bugs-527783 ] popen3 hangs on windows
noreply@sourceforge.net
noreply@sourceforge.net
Thu, 14 Mar 2002 12:22:27 -0800
Bugs item #527783, was opened at 2002-03-09 13:13
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=527783&group_id=5470
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Chris Withers (fresh)
Assigned to: Nobody/Anonymous (nobody)
Summary: popen3 hangs on windows
Initial Comment:
The following hangs on windows:
(i,o,e)=popen('python test.py')
result=o.read()+e.read()
...where test.py is the test.py of a Zope 3 CVS
checkout. I suspected a Zope 3 problem, but Thomas
Guettler also expereinced this in a different context:
popen3() of the python (2.1.2) which comes with zope
hangs on W2K:
(stdin, stdout, stderr)=popen3('wvWare -x wvware.xml
foo.doc')
text=stdout.read()
Then again, having seen bug #481896, I'm not sure this
is confined to windows.
Any ideas?
----------------------------------------------------------------------
>Comment By: Chris Withers (fresh)
Date: 2002-03-14 20:22
Message:
Logged In: YES
user_id=24723
This was an application problem, see:
http://mail.python.org/pipermail/python-dev/2000-September/009460.html
Here's my new code which solves the problem, maybe
NonBlockingReader belongs in a library somewhere?
cheers,
Chris
PS:
from os import getcwd, chdir, system, popen3
from threading import Thread
class NonBlockingReader(Thread):
def __init__(self,file):
Thread.__init__(self)
self.file = file
def run(self):
self.result = self.file.read()
def read(self):
return self.result
dir=getcwd()
chdir('E:\ZopeTests\sandbox\Zope3')
(i,c,e) = popen3('c:\python22\python test.py')
chdir(dir)
print "popened"
ct = NonBlockingReader(c)
print "ct created"
et = NonBlockingReader(e)
print "et created"
ct.start()
print "ct started"
et.start()
print "et started"
ct.join()
print "ct joined"
et.join()
print "et joined"
print ct.read()+et.read()
----------------------------------------------------------------------
Comment By: Darrell Gallion (dgallion)
Date: 2002-03-10 19:29
Message:
Logged In: YES
user_id=68151
http://ezwiki.com/2.54/python/popenTest.py
This test works on
Linux and Win2k.
It makes me think raw_input should flush
stdout.
It's tricky getting this sort of thing to work.
Looks like
test_popen2.py uses "more.exe" on Win98.
That might be a
problem?
The client program must flush stdout after each write.
----------------------------------------------------------------------
Comment By: Darrell Gallion (dgallion)
Date: 2002-03-10 17:27
Message:
Logged In: YES
user_id=68151
I don't know about Win98*. But the example
given did hang in Linux also.
The
problem as posted here seems to be caused
by fp.read() blocking until the spawned
program exits. Using os.read() on Linux
avoids this problem. I'll give it a try on
Win2k and XP today.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2002-03-10 09:19
Message:
Logged In: YES
user_id=31435
If would help a lot if someone produced a small, self-
contained test case that reproduced the problem. The usual
outcome of *trying* to is the discovery that popen() on
Windows hangs "for no identifiable reason at all".
Example: there's a test_popen2.py in the std Python test
suite. It hangs about 1 time in 200 when I run it on
Win98SE. There's no identifiable cause; it just hangs
sometimes, and when it does it's always hung in the bowels
of MS's code. There are no races in the Python code
driving it.
----------------------------------------------------------------------
Comment By: Darrell Gallion (dgallion)
Date: 2002-03-09 23:54
Message:
Logged In: YES
user_id=68151
os.read() doesn't block.
I'd expect you have to
call os.read in a loop.
i,o,e=os.popen3("../../python
errXX.py")
print os.read(o.fileno(),10000),
os.read(e.fileno(),10000)
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2002-03-09 18:22
Message:
Logged In: YES
user_id=31435
Chris, which version of Windows are you talking about? You
identified W2K for Thomas Guettler, but didn't say which
version you were using. The popen implementations are
radically different across Windows flavors, so knowing
which one you're using is important.
Alas, "important" doesn't necessarily imply helpful
<wink>. Historically, popen-on-Windows bugs don't get
fixed, because they hit a brick wall after blame gets
traced to MS internals.
----------------------------------------------------------------------
Comment By: Chris Withers (fresh)
Date: 2002-03-09 18:14
Message:
Logged In: YES
user_id=24723
So what should I use instead of e.read() ad o.read()
In fact, any chance of a replacement line for:
result=o.read()+e.read()
? ;-)
cheers,
Chris
----------------------------------------------------------------------
Comment By: Darrell Gallion (dgallion)
Date: 2002-03-09 18:11
Message:
Logged In: YES
user_id=68151
Don't use e.read()
This will try to read the entire file. Which doesn't
make sense with a stream. Although I think this works with sockets?
I'll look at a select solution.
Not sure how to know how many bytes to
read after the select breaks out.
----------------------------------------------------------------------
Comment By: Darrell Gallion (dgallion)
Date: 2002-03-09 14:51
Message:
Logged In: YES
user_id=68151
This cuased the problem on Linux.
import os, re
files="""
file
errXX.py
<<<<<<<<<<<<<
import sys
while 1:
print >>sys.stderr,
'x'
<<<<<<<<<<<<<
file runXX.py
<<<<<<<<<<<<<
import
os
(i,o,e)=os.popen3("python errXX.py")
print
e.read()
<<<<<<<<<<<<<
cmd python
runXX.py
<<<<<<<<<<<<<
"""
files=re.split("<<<<<+",files)
for
x in range(0,len(files), 2):
cmd=files[x].split()
body=files[x+1]
if cmd[0]=='file':
open(cmd[1],'w').write(body)
elif cmd[0]=='cmd':
os.system('
'.join(cmd[1:])+body)
else:
assert(0)
----------------------------------------------------------------------
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=527783&group_id=5470