[Python-bugs-list] [ python-Bugs-216289 ] Programs using Tkinter sometimes can't shut down (Windows)

noreply@sourceforge.net noreply@sourceforge.net
Fri, 16 Aug 2002 11:56:16 -0700


Bugs item #216289, was opened at 2000-10-06 19:25
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470

Category: Windows
Group: Python 2.3
Status: Open
Resolution: Later
Priority: 3
Submitted By: Tim Peters (tim_one)
Assigned to: Fredrik Lundh (effbot)
Summary: Programs using Tkinter sometimes can't shut down (Windows)

Initial Comment:
The following msg from the Tutor list is about 1.6, but I noticed the same thing several times today using 2.0b2+CVS.  In my case, I was running IDLE via

python ../tool/idle/idle.pyw

from a DOS box in my PCbuild directory.  Win98SE.  *Most* of the time, shutting down IDLE via Ctrl+Q left the DOS box hanging.  As with the poster, the only way to regain control was to use the Task Manager to kill off Winoldap.

-----Original Message-----
From: Joseph Stubenrauch <nothingisgoingtochangemyworld@yahoo.com>
Sent: Friday, October 06, 2000 9:23 PM
To: tutor@python.org
Subject: Re: [Tutor] Python 1.6 BUG


Strange, I have been experiencing the same bug myself.
Here's the low down for me:

Python 1.6 with win95
I am running a little Tkinter program
The command line I use is simply: "python foo.py"
About 25-35% of the time, when I close the Tkinter
window, DOS seems to "freeze" and never returns to the
c:\ command prompt.  I have to ctrl-alt-delete
repeatedly and shut down "winoldapp" to get rid of the
window and then shell back into DOS and keep working.
It's a bit of a pain, since I have the habit of
testing EVERYTHING in tiny little stages, so I change
one little thing, test it ... freeze ... ARGH!  Change
one more tiny thing, test it ... freeze ... ARGH! 
However, sometimes it seems to behave and doesn't
bother me for an entire several hour session of python
work.

That's my report on the problem.

Cheers,

Joe


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 11:56

Message:
Logged In: YES 
user_id=72656

I'm finding it a bit hard to reproduce, and now not at all.  
Could people please try replacing their current tcl83.dll with 
the one at http://www.tcl.tk/tcl83.dll.  The sig on it is:
634880 Aug 16 10:53 tcl83.dll
(md5) b35628568ddc4afed4f675d2d9a50faf  tcl83.dll

I can't hang anymore with python 2.2.1 at the Win98 
command prompt with:
   python \Python22\Tools\idle\idle.pyw

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 10:50

Message:
Logged In: YES 
user_id=72656

johnny - That actually helps a lot and fairly clearly indicates 
the source of this hanging problem.  To give more context to 
that area, this is in the final shutdown code where it is 
(obviously) shutting down the console I/O.  The reader is 
shutdown without a check for more input (why, we are 
exiting?), so there is no issue there.

The writer however wants to make sure that the console 
output channel is drained before exit.  A blocking channel 
that is not being flushed will cause a hang.

So the question is, what is the stdio setup for Tcl in 
Py/Tkinter?  That should be examined to see whether stdio is 
clear to flush on exit.

Perhaps you can also help with more debugging.  Could you 
wrap the problematic WaitForSingleObject call with
if (consolePtr->toWrite) { ...
and perhaps also print out what to write.  It may be that 
blocking output isn't the problem at all, but rather that we are 
never receiving the signal on consolePtr->writable (usually 
done in the writer thread).  The consolePtr->toWrite check 
will at least ensure flushed channels do not enter this 
potential hanging state.

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-16 07:16

Message:
Logged In: YES 
user_id=143340

I have dug out the setup I used previously and, apart from 
using Python 2.2 instead of 2.0.1, have had a go at 
reproducing my previous results.

With a debug build of Tcl/Tk and threads disabled I have 
managed to get this stack trace when it locks on exit (after 
doing a break):

ConsoleCloseProc(int * 0x006a0ae4, Tcl_Interp * 
0x00000000) line 527 + 17 bytes
TclFinalizeIOSubsystem() line 241 + 20 bytes
Tcl_FinalizeThread() line 911
Tcl_Finalize() line 812
DllMain(HINSTANCE__ * 0x00970000, unsigned long 
0x00000000, void * 0x00000001) line 197
TCL83! _DllMainCRTStartup@12 + 80 bytes
KERNEL32! bff7ddd6()
KERNEL32! bff8d123()
8176b5fc()

It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
at about line 527:
WaitForSingleObject(consolePtr->writeThread, INFINITE);

I tried changing the timeout from INFINITE to 100ms and 
testing the return value for WAIT_TIMEOUT. Sure enough it 
occasionaly printed a message (and didn't lock-up).

Don't know if this helps. Will try to reproduce the double 
free problem in the threaded build later,

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-15 07:19

Message:
Logged In: YES 
user_id=6380

Just a note so I won't forget this: Jeff Hobbs once said the
code to look at is

tk/generic/tkConsole.c:Tk_InitConsoleChannels

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-15 06:23

Message:
Logged In: YES 
user_id=7549

Hi John,

I have seen Tcl extension DLLs that set exit handlers get 
called by Tcl after they were torn-down.  NT and Win9x seem 
to be a bit different in the order of what DLLs get torn-down.  
here's a note from some old code of mine:

#ifdef __WIN32__
    // It can happen that the HEAP could have already been 
unloaded
    // from an awkward teardown caused by a Ctrl+C or other.  
Win32
    // seems to do a teardown in reverse order and by the time 
Tcl
    // knows what's going on and Tcl_Finalize calls the exit
    // handlers, this extension's data (heap?) has already been
    // unloaded by the OS.  Do a quick legal check on the 
pointer
    // first.
    //
    if (IsBadReadPtr(adapt, sizeof(T *))) return;
#endif

what's the back/stacktrace?  who is calling 
Tcl_FinalizeNotifier?  And had anyone called it before during 
the shutdown phase? I can't say with much confidence that 
this is happening here, too.  Maybe..  Could Tk be telling Tcl 
shutdown?  That's sort of backwards, if so.

I run w2k here, so I can't help in debugging from a test case.  
tclsh with a threaded build doesn't even call Tcl_Finalize to 
avoid all this!

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2002-08-15 06:09

Message:
Logged In: YES 
user_id=79902

Firstly, on the matter of threads.  Tcl/Tk on Windows uses
threads internally even in non-threaded builds.  That's
Windows for you (in particular, you can get calls parachuted
in from completely separate threads with relatively little
warning.  Also, you need threaded helpers to handle
non-blocking waiting on some kinds of objects.)

It's a bit hard to work out if the problem's been fixed
without a stack-trace (preferably with debugging symbols
enabled so that we can see what the function names and
arguments to those functions are.)  I have no  chance of
replicating this bug here (I'm currently running IRIX, but
also have some access to Solaris and Linux...)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-15 05:15

Message:
Logged In: YES 
user_id=143340

Hi to davygrvy,

my fix isn't for a thread related (deadlock) type problem - it 
is a fix for a double free type problem which is presumably 
screwing up the heap and causing the C runtime to hang.

It *is* with a threaded build (THREADDEFINES = -
DTCL_THREADS=1) which I think I enabled because I was 
using  'after' and it didn't work unless threading was 
enabled - but I may be mis-remembering here!

Also, I ended up doing all this with the debug build and 
found the -GZ flag useful - in fact it pointed out the fact that 
there was a problem with the heap.

It is worth noting that I have only seen this on Win98 and 
that using 'pythonw.exe' did *not* prevent this from 
happening,

regards John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-14 18:00

Message:
Logged In: YES 
user_id=6380

AFAIK we never distributed a threaded build.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-14 17:01

Message:
Logged In: YES 
user_id=7549

To johnnypops,

Where in Tcl_FinalizeNotifier() is the dead-lock occuring and 
is this with a threaded build of Tcl?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-02 12:39

Message:
Logged In: YES 
user_id=31435

Jeff, a long time ago Fredrik Lundh said he was able to 
reproduce this with a short wish script (no Python, no Tkinter, 
just Tcl/Tk + wish).  I'm reassigning this to /F in the hopes 
he'll remember how.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-07-02 11:55

Message:
Logged In: YES 
user_id=72656

I appreciate that many others see this is a problem in 
embedding Tcl, but without any clear way to reproduce this in 
Tcl/Tk itself, it's very hard for me to fix it.  I don't think that 
there is anything done in 8.4 to fix this that I recall, but if 
someone finds that the problem "goes away" with 8.4, please 
report that.

For those of course that experience this with python, the 
simple and "correct" work-around is to launch with 
pythonw.exe instead of python.exe.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-26 10:40

Message:
Logged In: NO 

According to http://tcl.activestate.com ActiveTcl 8.4.0 is now 
available for download.  It is new as of June 24.  I don't know 
whether this will solve our problems or not, but it's worth 
looking into.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-19 06:36

Message:
Logged In: NO 

I just downloaded Ruby 1.6.6 for Windows, which also uses 
Tcl/Tk 8.3 and I experience exactly the same problem there. 
So this is definitely a Tcl problem (but we know that already).

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-06-07 12:01

Message:
Logged In: YES 
user_id=143340

Hmmm. 
I run a Python Tcl/Tk application several times every day - 
the very application that prompted my bug-hunt.

I do use "pythonw.exe" to launch it, but it used to hang 
almost as badly as "python.exe".

Whilst developing this app I used "python.exe" and 
experienced no hang-ups.

The Tcl guys point out that this bug is only one of several 
possible causes of the hang-up, maybe you are just out of 
luck.

The patch has fixed the problem on three Win98 machines, 
all of which exhibited the problem before updating the DLLs.

One thought - are you sure there weren't multiple copies of 
the tcl83.dll and tk83.dll files? My (only) copies were 
in "D:\python20\DLLs"

good luck,
John.


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-06-03 11:55

Message:
Logged In: YES 
user_id=72656

providing Tim with a fixed DLL exactly according to John 
Popplewell's patch did not fix the problem for him.  That 
makes it seem fishy that John really didn't see the problem 
any more.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-03 09:59

Message:
Logged In: YES 
user_id=31435

I tried replacing Python 2.2.1's tk83.dll and tcl83.dll (from 
Tcl/Tk 8.3.2) with newer matched sets from

A) PythonWare's py21 distribution (Tcl/Tk 8.3.3).

and again from

B) ActiveState's Tcl/Tk 8.3.4.

All the symptoms originally reported here persisted on 
Win98SE despite that.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-05-22 06:57

Message:
Logged In: NO 

I am new to Python and encountered this bug on my very first 
Tkinter application.  It persists in Python 2.2.1 under 
Windows Me. Needless to say, it makes a bad first 
impression.  Basically, this thread says that Tkinter has been 
broken for a year and a half on a popular platform at a time 
when Python is trying to expand its user base.  Hopefully, this 
bug will be corrected soon.

If Tcl/Tk 8.4 is not released shortly you might try repackaging 
Tkinter for Win32 with an earlier version of Tcl/Tk, such as 
8.0, that doesn't encounter this problem.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 19:36

Message:
Logged In: YES 
user_id=6380

The proper action is to wait until Tcl 8.3.5 is released,
and then shaming someone else in providing you with a set of
Windowsbinaries for Tcl. You may also ask Hobbs if there's a
Windows project file to build Tcl/Tk for Windows.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-03-11 18:31

Message:
Logged In: YES 
user_id=72656

I did end up back-porting this to the 8.3.4+ Tcl CVS on 
2002-02-25, which means it will be in an eventual 8.3.5.  
It is already in the release 8.4 alpha series.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-09 17:50

Message:
Logged In: YES 
user_id=31435

Back to Guido:  Do you think you know what the proper 
action is?  (You said that's why you reopened it, and 
there's been no new info here since then.)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 16:55

Message:
Logged In: YES 
user_id=143340

I knew I wasn't getting to the heart of it ....

Almost a one-liner!

It has been seriously spoiling my (otherwise crash free) 
Python experience on Win9x - hope it gets fixed soon.

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-25 15:39

Message:
Logged In: YES 
user_id=6380

Reopened until we know what the proper action is.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-02-25 15:32

Message:
Logged In: YES 
user_id=72656

This is mostly correct, and is fixed in the 8.4 Tcl sources 
already (I guess we can backport this).  This was mentioned 
in SF Tcl bug (account for chopped URL):

https://sourceforge.net/tracker/?
func=detail&aid=217982&group_id=10894&atid=110894

and the code comment is:

    /*
     * Only finalize the notifier if a notifier was 
installed in the
     * current thread; there is a route in which this is not
     * guaranteed to be true (when tclWin32Dll.c:DllMain() 
is called
     * with the flag DLL_PROCESS_DETACH by the OS, which 
could be
     * doing so from a thread that's never previously been 
involved
     * with Tcl, e.g. the task manager) so this check is 
important.
     *
     * Fixes Bug #217982 reported by Hugh Vu and Gene 
Leache.
     */
    if (tsdPtr == NULL) {
	return;
    }


----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 14:41

Message:
Logged In: YES 
user_id=143340

This one has been torturing me for a while.

Managed to track it down to a problem inside Tcl.
For the Tcl8.3.4 source distribution the problem is in
the file win/tclWinNotify.c 

void Tcl_FinalizeNotifier(ClientData clientData)
{
    ThreadSpecificData *tsdPtr = 
                         (ThreadSpecificData *) clientData;

    /* sometimes called with tsdPtr == NULL */
    if ( tsdPtr != NULL )
    {
        DeleteCriticalSection(&tsdPtr->crit);
        CloseHandle(tsdPtr->event);

        /*
         * Clean up the timer and messaging 
         * window for this thread.
         */

        if (tsdPtr->hwnd) {
	    KillTimer(tsdPtr->hwnd, INTERVAL_TIMER);
            DestroyWindow(tsdPtr->hwnd);
        }
    }

    /*
     * If this is the last thread to use the notifier, 
     * unregister the notifier window class.
     */

    Tcl_MutexLock(&notifierMutex);
    if ( notifierCount && !--notifierCount ) {
	UnregisterClassA( "TclNotifier", 
                          TclWinGetTclInstance());
    }
    Tcl_MutexUnlock(&notifierMutex);
}

This bodge doesn't address the underlying problem but has 
stopped me from tearing all my hair out,

cheers,
John Popplewell.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-10-24 15:27

Message:
Logged In: YES 
user_id=31435

FYI, you don't need an IDE to do this -- in Win9x, hit 
Ctrl+Alt+Del and kill the process directly.

A saner <wink> solution is to develop under Win2K, which 
doesn't appear to suffer this problem (the only reports 
I've seen, and experienced myself, came from Win9x boxes).

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-10-24 01:52

Message:
Logged In: NO 

For those who are still trapped in this bug's hell, I will 
gladly share the one thing that saved my sanity:  
WingIDE's 'Kill' command will shut down the program with 
apparent 100% certainty and no fear of lockups.  WingIDE 
has its own issues, so its not a perfect solution, but if 
you are like me and Joe (above) who test in small 
iterations, then using 'Kill' to quit out of your app while 
testing is a workaround that may preserve your sanity.

Perhaps the python gods and the Wing guys can get together 
and tell us how to replicate 'kill' into our code.  For 
now, I'll use WingIDE to edit, and pythonw.exe for my final 
client's delivery.

----------------------------------------------------------------------

Comment By: Howard Lightstone (hlightstone)
Date: 2001-09-05 10:43

Message:
Logged In: YES 
user_id=66570

I sometimes get bunches of these....

A tool I use (Taskinfo2000) reports that (after killing 
winoldap):

python.exe is blocked on a mutex named OLESCELOCKMUTEX.
The reported state is "Console Terminating".  There appears 
to be only one (os) thread running.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-04-02 13:06

Message:
Logged In: YES 
user_id=31435

No sign of progress on this presumed Tk/Tcl Windows bug in 
over 3 months, so closing it for lack of hope.

----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2001-02-05 21:13

Message:
This was a symptom I saw while tracking down the essence of the problem reported in #131207.  Using Win98SE, I would get an error dialog (GPF?) in the Kernel, and sometimes the dos prompt would not come back.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-12-12 18:00

Message:
Just reproduced w/ current CVS, but didn't hang until the 8th try.

http://dev.scriptics.com/software/tcltk/ says 8.3 is still the latest released version; don't know whether that URL still makes sense, though.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-12-12 12:58

Message:
Tim, can you still reproduce this with the current CVS version?

There's been one critical patch to _tkinter since the 2.0 release.

An alternative would be to try with a newer version of Tcl (isn't 8.4 out already?).


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2000-10-15 09:47

Message:
Same as I've reported earlier; it hangs in the call to
Tcl_Finalize (which is called by the DLL finalization
code).  It's less likely to hang if I call Tcl_Finalize
from the _tkinter DLL (from user code).

Note that the problem isn't really Python-related -- I
have stand-alone samples (based on wish) that hangs in
the same way.

More later.

</F>

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-13 07:40

Message:
Back to Tim since I have no clue what to do here.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-12 10:25

Message:
The recent fix to _tkinter (Tcl_GetStringResult(interp) instead of interp->result) didn't fix this either.

As Tim has remarked in private but not yet recorded here, a workaround is to use pythonw instead of python, so I'm lowering thepriority again.

Also note that the hanging process that Tim writes about apparently prevents Win98 from shutting down properly.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-10-07 00:37

Message:
More info (none good, but some worse so boosted priority):

+ Happens under release and debug builds.

+ Have not been able to provoke when starting in the debugger.

+ Ctrl+Alt+Del and killing Winoldap is not enough to clean everything up.  There's still a Python (or Python_d) process hanging around that Ctrl+Alt+Del doesn't show.

+ This process makes it impossible to delete the associated Python .dll, and in particular makes it impossible to rebuild Python successfully without a reboot.

+ These processes cannot be killed!  Wintop and Process Viewer both fail to get the job done.  PrcView (a freeware process viewer) itself locks up if I try to kill them using it.  Process Viewer freezes for several seconds before giving up.

+ Attempting to attach to the process with the MSVC debugger (in order to find out what the heck it's doing) finds the process OK, but then yields the cryptic and undocumented error msg "Cannot execute program".

+ The processes are not accumulating cycles.

+ Smells like deadlock.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470