[Idle-dev] Leaving subprocesses hanging

[Kurt B. Kaiser]

John Zelle <john.zelle at wartburg.edu> writes:

> I'm new to the idle-dev list, but felt I needed to post here for some
> assistance. I am in communication with a number of college faculty
> using Python to teach introductory programming courses. There has been
> growing frustration that newer versions of IDLE seem to be less
> stable. In particular, especially under Windows XP, there are times
> when IDLE becomes unresponsive (hangs). This seems to be related to
> losing proper communication with its subprocess. The only way to get
> things cleaned up again is to go into the process manager and kill the
> stray pythonw processes. Unfortunately, the behavior is sporadic and
> hard to pin down.

This is the major difficulty with using subprocess execution on
Windows.

I spent quite a bit of time on the 'hanging subprocess' bug a year and
a half ago.  The bug was so persistent I've named it 'Freddy'.

My priority with IDLE is to choose stability over bells/whistles.
IDLE is intended to be used for introductory programming as well as by
advanced programmers, but beginners take precedence in my view.

At one time (May '03), I was interrupting the subprocess before
restarting it but the resulting traceback was slowing down the restart
considerably.  So I redesigned the subprocess so that a loss of socket
connection to the GUI would invoke an interrupt of the main thread
(which is executing the user's code).  The GUI causes the subprocess to
restart by simply closing the socket.  On Unix, a TERM signal is also
sent to the subprocess, but that's not available on Windows.

> I did a little poking around in the idle-dev archives and did not
>see anything about this problem, so I did a bit of experimenting on
>my own. One simple case that causes IDLE to hang is to simply close
>the shell while the program that it is executing is waiting for
>input. After IDLE asks if it should kill the program, it usually
>hangs. If it doesn't do this the first time, a couple of <f5>-close
>sequences will produce it.

OK, so I assume you are using something like

.>>> a = raw_input('prompt: ')

and then either clicking the shell window close widget or using F5.
That's actually two different scenarios.  I'll go off and duplicate
them.

> I modified PyShell.py in IDLE 1.03 by adding a call to
> cancel_callback() before closing the shell window. This seems to
> solve the problem (perhaps in an inelegant way). 

Where, exactly, did you put this code?  Did it fix the problem
you mentioned with <F5>-close sequences?

> I do not know yet if fixing this "special" case solves the larger
> instability problems. 

Shutting down the subprocess on Windows involves having it detect the
loss of connection and then successfully shut down the main thread,
the socket thread, and any threads associated with GUI communication.
It's complicated, and 'fixes' tend to regress older issues.

> I'm hoping some of the reader of edu-sig are trying it out and will
> let me know.
>
> In the mean time, I'd appreciate hearing if this is a known problem
> that is being worked on in IDLE. My preliminary test of IDLE 1.1 shows
> that it still suffers from the specific hang that I've identified.

It's an old problem which I thought was fixed since it hasn't been
reported recently.

I hope that edu-sig and c.l.p will bring stability issues to this list
if they can't enter them on the Python Bug tracker.

I'm also interested in discussing IDLE usability issues, especially
in an educational context.  That's appropriate for this list.

-- 
KBK