[Idle-dev] [ idlefork-Bugs-677263 ] On W32 Can't Terminate Subprocess Not Doing IO

Sat, 17 May 2003 15:49:59 -0700

Bugs item #677263, was opened at 2003-01-29 23:02
Message generated for change (Comment added) made by kbk
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=109579&aid=677263&group_id=9579

Category: None
Group: None
Status: Open
Resolution: Fixed
Priority: 8
Submitted By: Bruce Sherwood (bsherwood)
Assigned to: Kurt B. Kaiser (kbk)
Summary: On W32 Can't Terminate Subprocess Not Doing IO

Initial Comment:
With 0.9a2 it is still the case that something is very 
wrong with the run environment for VPython programs, 
at least on Windows. I run the simplest program

from visual import *
sphere()

Next I kill the graphics display window (by clicking in the 
close box), then type in the shell "2+2". I get the 
following:

Traceback (most recent call last):
  File "C:\Python23\Lib\site-packages\idleforklib\rpc.py", 
line 187, in asynccall
    self.putmessage((seq, request))
  File "C:\Python23\Lib\site-packages\idleforklib\rpc.py", 
line 289, in putmessage
    n = self.sock.send(s)
  File "<string>", line 1, in send
error: (10054, 'Connection reset by peer')

I then try to kill the shell window and get a dialog box 
asking this:

"The program is still running; do you want to kill it?"

I say I want to kill it, the dialog box goes away, but 
nothing happens to the shell window. The only way I can 
kill Python is with the Windows task manager. Even 
independent of what may be special about a VPython 
program, shouldn't IDLE honor the request to kill?

I emphasize that with the original idlefork when one 
killed a VPython graphics window you were in a good 
state. Another possible hint at the source of the new 
problem is that with the original idlefork after F5 to run I 
could press F5 and the existing graphics window would 
be killed and I'd get a new spawned process. Now I 
seem to get two spawned processes in this situation.

http://vpython.org is a place to get the VPython stuff. I 
have no idea whether the problem is in some sense with 
VPython or with the new idlefork.

----------------------------------------------------------------------

>Comment By: Kurt B. Kaiser (kbk)
Date: 2003-05-17 17:49

Message:
Logged In: YES 
user_id=149084

BTW,  with the original IDLEfork the subprocess
terminates when the user code runs off the end.
In the current implementation it doesn't; the shell
stays connected to the subprocess so you can examine
the final environment.  That's why you only see the
one pythonw.exe in the original version.

----------------------------------------------------------------------

Comment By: Kurt B. Kaiser (kbk)
Date: 2003-05-17 16:15

Message:
Logged In: YES 
user_id=149084

I've checked in run.py 1.18, give it a try (though I don't
have much hope...)

I imagine you are starting up with just an edit window.
In that case you will see one pythonw.exe.  Once you
run the program the GUI will open a shell/output window
and spawn the subprocess.  Every time you re-run the
code the subprocess is terminated and a new one spawned.
So what you normally see from then on is two pythonw.exe.
Nothing to be alarmed about.  Now, if for some reason the
MainThread can't exit on Windows the GUI process loses
track of it and it just sits there and spins.  Every time the 
subprocess is restarted, another pseudo-zombie is created.

On W2K I can do all sorts of terrible things: put the 
subprocess into a pass loop or a print loop and then kill
the GUI with the task manager.  When I do that, the
subprocess exits OK.  But on your system, it seems
the MainThread is getting blocked somehow.

Try the new run.py and let me know.  If it doesn't work,
I have a nice fast XP box I built a while ago which I 
haven't had time to install Python on.  

Thanks for spending the time to help on this!  Freddy
has been holding up the release for a month.

----------------------------------------------------------------------

Comment By: Bruce Sherwood (bsherwood)
Date: 2003-05-17 12:59

Message:
Logged In: YES 
user_id=34881

One other comment: With the original idlefork, running my 
trivial program (print 2+2) brings up a 2nd pythonw.exe very 
briefly, but it then quickly goes away as soon as the"4" is 
printed, leaving only one copy active. It is somewhat alarming 
that with the new idlefork there are THREE copies of idlefork, 
and neither of the additional two copies goes away after 
the "4" is printed.

----------------------------------------------------------------------

Comment By: Bruce Sherwood (bsherwood)
Date: 2003-05-17 12:54

Message:
Logged In: YES 
user_id=34881

I replaced os._exit(0) with return, and that didn't fix it. I also 
installed idlefork with Python 2.2 and verified that behavior is 
identical on 2.2 and 2.3 on Windows XP. 

More details: It takes only a few seconds for the shell/output 
window to open (corresponding to the 2nd copy of 
pythonw.exe starting to run). It takes another 30 seconds 
before I see the output of my tiny program (print 2+2), and 
this corresponds to seeing in the system monitor the 3rd 
copy of pythonw.exe very slowly starting up, with its memory 
allocation growing very slowly until at about 5.7 MB I see my 
output. Thereafter CPU usage is pinned at 100%. I then quit 
either by clicking the close boxes on the two windows or by 
choosing the Exit menu option in one of the windows. The 
first pythonw.exe quits, but the 2nd and 3rd copies stay up 
and consume 100% of the CPU. I have to kill them in the 
system monitor. Now what? 

Note that it isn't just the quitting that fails catastrophically. 
There are two other very serious problems: It takes a very 
long time to get output (30 seconds), and there is 100% CPU 
usage even before I try to quit the program.

----------------------------------------------------------------------

Comment By: Kurt B. Kaiser (kbk)
Date: 2003-05-17 10:05

Message:
Logged In: YES 
user_id=149084

I assume you can see this if you just restart the shell
without running any program.

CVS works fine on W2K. But if I replace run.py:56 os._exit(0) 
with sys.exit(0), then the subprocess hangs as you are
seeing.  That's why I used os._exit.  It appears 
something changed between Py2.2/2.3 and/or W2K/XP and
os._exit() on your box is acting like sys.exit() on my setup.

The way this works is when SockThread gets an EOF
due to the link being closed by the GUI, it sets the global
exit_requested and goes into a pass loop waiting for the
MainThread to terminate the whole process.  Every 10
bytecodes the MainThread gets control and checks 
exit_requested.  If True, it exits and is supposed to take
the SockThread with it since it's setDaemon.  But if the
MainThread exits without shutting down SockThread the
latter just spins, taking 100%, which is what you are 
seeing.  If I exit the SockThread instead of waiting, there
are other problems due to the Server objects disappearing
when then are still needed by MainThread and its various
socket threads (Thread-1, Thread-2 etc) not having
wrapped up communication.  That also causes hangs.

Try this, it works on my system: 
replace 
run.py:56 os._exit(0) 
with 
run.py:56 return.  

We'll let Python decide how to exit.

----------------------------------------------------------------------

Comment By: Bruce Sherwood (bsherwood)
Date: 2003-05-17 08:26

Message:
Logged In: YES 
user_id=34881

I do have run.py Rev 1.17. I got everything from CVS on 5/16. 
And I've checked by hand that I do have the addition 
of "except KeyboardInterrupt" in my copy of run.py. So that's 
not the explanation. What should I try now?

----------------------------------------------------------------------

Comment By: Kurt B. Kaiser (kbk)
Date: 2003-05-16 17:15

Message:
Logged In: YES 
user_id=149084

It's critical that you have run.py Rev 1.17
which was checked in 2003/05/14 18:15:40to fix this
problem.

The relevant line is run.py:65  except KeyboardInterrupt:
This line and the except clause following must be present.
Without this, the restart doesn't work because the
subprocess SockThreadstops servicing the socket while
the MainThread is trying to print an exception.

I was seeing the same before the fix.  Freddy's baaack?

----------------------------------------------------------------------

Comment By: Bruce Sherwood (bsherwood)
Date: 2003-05-16 16:44

Message:
Logged In: YES 
user_id=34881

On Windows XP I installed Python 2.3 and idlefork from CVS. 
I see massive problems. Minimal program "print 2+2" takes 
30 seconds or more after pressing F5 before I see the output 
(consuming 100% of the CPU all the while), and even after 
seeing the output the system monitor shows me consuming 
100% of the CPU. I then close both windows (source and 
output), and I'm left with two instances of python.exe which 
together or singly consume 100% of the CPU, and I have to 
kill them in the system monitor. Is there something I'm 
missing about installation? I have the idlefork package in 
C:\Python23\idle. I did run C:\Python23\python.exe setup.py 
install inside the idle directory in order to get the interrupt 
module compiled. I assume that I have all the pieces in that I 
did get output.

----------------------------------------------------------------------

Comment By: Kurt B. Kaiser (kbk)
Date: 2003-05-14 22:48

Message:
Logged In: YES 
user_id=149084

Please try current CVS and report any problems.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2003-04-01 00:51

Message:
Logged In: NO 