[Python-bugs-list] [ python-Bugs-694062 ] os.popen() hangs on {Free,Open}BSD

SourceForge.net noreply@sourceforge.net
Tue, 04 Mar 2003 12:45:52 -0800


Bugs item #694062, was opened at 2003-02-27 02:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694062&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Jason R. Mastaler (jasonrm)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.popen() hangs on {Free,Open}BSD

Initial Comment:
I'm seeing odd behavior with os.popen() on 
certain platforms such as FreeBSD and OpenBSD. 
 
FWIW, I posted about this problem almost two 
years ago on c.l.p, and never got a resolution. 
 
See 
http://mail.python.org/pipermail/python-list/2001-May/042373.html 
 
Now, many Python and FreeBSD upgrades later, 
I see the problem still exists. Now however, I need to 
find a solution as I can't workaround it any longer. 
 
I've attached about the simplest test case I could 
produce below. You can call it from a .forward or 
.qmail file.  
 
On Free/OpenBSD, the program runs but leaves a  
python process lying around: 
 
jasonrm    142  0.0  0.2  2576 2008  ??  I     5:42PM   
0:00.02 python /tmp/hang (python2.2) 
 
On other platforms such as Linux, the program also 
runs, but no process is left hanging around.  
 
I've been able to reproduce the problem on Free/OpenBSD 
only. The problem doesn't appear on IRIX, BSD/OS or  
Linux. 
 
Please let me know if I can do anything more to help 
debug this.  

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-03-04 21:45

Message:
Logged In: YES 
user_id=45365

That confirms my suspicion that this is a third-party error: filedescriptors 2 and 3 are readwrite on BSD, write-only on Linux.

A final test to confirm this: on BSD, add code like this just before calling popen():
os.close(1)
os.close(2)
os.open("/dev/null", "w")
os.open("/dev/null", "w")

If this makes the hang go away then the real problem is that whatever program executes your .forward (qmail?) erroneously opens a read-write pipe for your stdout and stderr. You can't do this with pipe() AFAIK, but maybe with socketpair() or something.

----------------------------------------------------------------------

Comment By: Jason R. Mastaler (jasonrm)
Date: 2003-03-04 19:45

Message:
Logged In: YES 
user_id=85984

Attached is lsof output on a Linux box. I had to insert
a sleep after the write but before the close in order to
capture it.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-03-04 11:33

Message:
Logged In: YES 
user_id=45365

Something like fstat is indeed what I was suggesting. The fstat output shows your Python script is getting stdin from a file, and stdout/stderr go to a pipe (the same one). You also have another pip open on filedescriptors 3/4.

Try the same thing on Linux (or another system where things just work) and see if there's a difference in what files are open. 

----------------------------------------------------------------------

Comment By: Jason R. Mastaler (jasonrm)
Date: 2003-03-03 21:05

Message:
Logged In: YES 
user_id=85984

> Try finding out what stdin/stdout/stderr refer to 
> (with fstat or something).

Can you be more specific here?

I've attached the output of `fstat -m -p 15437' below
in case that helps.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-03-03 11:57

Message:
Logged In: YES 
user_id=45365

Just for fun I tried it on MacOSX, but running from the shall, not qmail, and I don't see any problems there. At least, not if I pass a small input message. If I pass a large input message I get a "broken pipe", but that is to be expected (as true doesn't really read it's input).

My guess is that this is a third-party error, somehow related to how qmail sets up stdin/stdout/stderr (apparently slightly different on Free/OpenBSD and the other systems). Try finding out what stdin/stdout/stderr refer to (with fstat or something). 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694062&group_id=5470