[Python-Dev] Tcl, Tkinter, and threads

Martin v. Löwis martin@v.loewis.de
12 Dec 2002 19:18:37 +0100


I have now applied changes to _tkinter to make it work when Tcl was
built with --enable-threads. For those of you interested in such kind
of stuff, here is the full story.

This work was triggered by the bug report

http://bugs.debian.org/171353

where pydoc -g crashes Tcl. It turns out that pydoc uses threads, and
invokes a button configure command from a thread different from the
one where the Tcl interpreter was called. Tcl uses thread-local
storage to associate a TkDisplay* with a X Display*, and that TLS was
uninitialized in this other thread.

Discussion with Tcl people at

http://sourceforge.net/tracker/index.php?func=detail&aid=649209&group_id=12997&atid=112997

lead to the conclusion that this is intentional; they call it the
"appartment model"; you can use the Tcl interpreter only from the
thread that has created it, you cannot pass Tcl_Obj* across thread
boundaries, and you cannot invoke Tcl commands from other threads.

If Tcl is not built for threads, this all is not relevant, since there
is only a single set of "thread-local" data (i.e. it isn't thread-local);
the thread API is still there, anyway.

To overcome this limitation, I now use Tcl Queues to pass commands
from one thread the other; this is also how the Tcl thread extension
works. You allocate a Tcl_Event, fill some data, put it into a queue,
alert the target thread, and block on a Tcl_Condition. The target
thread fetches the event from the queue, invokes the callback
function, which performs the Tcl action, and notifies the condition.

Passing of results happens to stack variables in the calling thread
whose addresses are put into the event.

This kind of marshalling now happens for the following APIs:
- call: passes the PyObject* args to the target thread. There it
  is converted into Tcl objects, the command is invoked, and the result
  is converted back to a Python object.
- getvar/setvar/unsetvar: pass the variable name and value, and invoke
  the Tcl API in the target thread.
- createcommand/deletecommand: likewise.

For a few APIs, this marshalling is not possible in principle:
- mainloop/doonevent: event processing must happen in the interpreter
  thread, by nature of the Tcl threading model

For a larger number of APIs, I found the effort not worthwhile:
- globalcall, eval, globaleval, evalfile, record, adderrorinfo,
  exprstring, exprlong, exprdouble, exprboolean, createfilehandler,
  deletefilehandler

For all these functions, _tkinter will now raise an exception if they
are invoked in the wrong thread.

A tricky question is what happens if the target thread is not
processing events right now, either because it mainloop hasn't been
invoked, or because it is busy doing something else. The current code
raises an exception if the target interpreter is not in the mainloop,
and blocks (potentially indefinitely) if the target process does not
unqueue its events.

This might cause problems if you create multiple interpreters in one
thread: it would be sufficient if one of them processes
events. Currently, calls to the other interpreters will raise the
exception that the mainloop has not been entered. I hope this won't
cause problems in practice, since you rarely have more than one
interpreter.

With these changes, it would now be possible to build Tcl in threaded
mode on Windows. This has both advantages and disadvantages:
+ It allows to get rid of the Tcl lock, and the nearly-busy wait.
- It may cause problems for existing applications if they run into
  one of the limitations. Of course, those applications will run into
  the same limitations on Unix if Tcl was build with threads enabled.

If Tcl wasn't build with threads enabled, behaviour is nearly
unmodified.  Python will still invoke the Tcl thread API, but that
won't do anything, so there should be no user-visible change.

Regards,
Martin