[Python-bugs-list] [ python-Bugs-226766 ] popen('python -c"...."') tends to hang

Mon, 24 Sep 2001 16:20:01 -0700

Bugs item #226766, was opened at 2000-12-24 08:08
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=226766&group_id=5470

Category: Windows
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: michal wallace (sabren)
Assigned to: Mark Hammond (mhammond)
Summary: popen('python -c"...."') tends to hang

Initial Comment:
eg:

import os
os.popen('python -c"x=1;print x"').readlines()

.. On my machine, using popen to call a second instance of python almost always causes python to freeze. No window pops up, but if I press alt-tab, there's an icon for w9xpopen.exe

oddly:

>>> os.popen('python -c"print"').readlines()

and

>> os.popen('python -c""').readlines()

both work fine.

... This bug is different from #114780 in that it is repeatable and consistent.

It happens on open, so is different from #125891. Eg:

>>> proc = os.popen('python -c"x=1; print x"')

will cause the crash.

----------------------------------------------------------------------

Comment By: Robert Amesz (rcamesz)
Date: 2001-09-24 16:20

Message:
Logged In: YES 
user_id=313190

Ah, well, it would seem that this is in fact the same bug I 
reported much later as #457466: "popenx() argument mangling 
hangs python". For some reason I overlooked it when posting 
my report, otherwise I'd have added my comments here. Sorry 
to add to the confusion. If you need to work around this bug 
(it's basically a bug in the commandline parsing), my report 
has information on that, as wel as a possible way of fixing 
it.

My patched version of popen9x.exe does not hang on Win98, 
but 
'reports' the error (reformatted to fit the margins):

Python 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit 
(Intel)] on win32 Type "copyright", "credits" or "license" 
for more information.
>>> import os
>>> os.popen('python -c"x=1;print x"').read()
"ARGUMENT PARSE ERROR\nThe way double quotes and spaces were 
used\ncaused popenx () to split the commandline.\n\nArgument 
count = 3\nArgument 0 = 
'C:\PYTHON21\w9xpopen.exe'\nArgument 1 = 'C:\COMMAND.COM 
/c python -cx=1;print'\nArgument 2 = 'x'\n"
>>>

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2000-12-29 03:29

Message:
um.....

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2000-12-28 15:34

Message:
I have done lots of playing with this over the last month or so.  The problem appears to be something to do with "python.exe" used as the target of such a popen command.  (Ironically, we tried to use popen to capture remote Python invocations for Komodo - as I guess you are for IDLE)

I have experimented with 3 "different" popen implementations: Python's, one written in Python using the win32 API directly, and one using the Netscape NSPR libraries.  They all, however, end up doing the same basic thing with the same basic Windows API functions.  They all behace the same WRT reading input.

Python.exe (and one or 2 other exes) appear to hang when they are in an "interactive loop", and the spawning process is trying to read the input pipe.  My experiements at breaking into the debugger shows Windows blocked inside the ReadFile() function.

Note that Perl.exe does _not_ appear to provoke this (but Perl doesnt have a builtin interactive loop, so is harder to prove).  Also note that "cmd.exe" also does _not_ provoke this - ie, both Perl.exe and cmd.exe both work fine, correctly reading and writing either pipe when the process is "interactive"

So - after all this, my best guess was that there is something weird in the stdio Python uses to read stdin, and that Perl and cmd.exe both avoid this.  Random, 100% speculative guess is that the "/MD" stdio is broken, but others wont be.

This is as far as I got - I was going to experiment with /MD, other input techniques etc, but never got to it.  It should be simple to reproduce with a simple test .exe, but alas never got to that either.  I should have more time over the next few weeks.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-12-28 07:47

Message:
Any chance you have Norton AntiVirus 2000 running?

See http://sourceforge.net/bugs/?group_id=5470&func=detailbug&bug_id=114598

I've basically given up on popen for windows. :-(

os.spawn*() works great though -- if you don't need to read the output. :-)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-12-24 09:40

Message:
Mark, any idea?  The first example also appears to hang for me consistently (W98SE).  In a debug build under the debugger, breaking during the hang yields a gibberish disassembly window (i.e., it's not showing code!), so I didn't get anywhere after 5 minutes of thrashing.

----------------------------------------------------------------------

Comment By: michal wallace (sabren)
Date: 2000-12-24 08:11

Message:
.. er.. whoops.. It hangs/freezes, not crashes. And in fact, it occasionally returns control to python after several minutes.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=226766&group_id=5470