[Python-Dev] Re: Extension modules, Threading, and the GIL

Mark Hammond mhammond@skippinet.com.au
Sat, 11 Jan 2003 11:22:28 +1100


[David]

> "Mark Hammond" <mhammond@skippinet.com.au> writes:
>
> > We could simply have a "pluggable TLS" design.
> >
> > It seems that everyone who has this requirement is interfacing to a
> > complex library (com, xpcom, Boost, ACE), and in general, these
> > libraries also require TLS.
>
> Boost isn't in that category.  Boost provides a threading library to
> establish a platform-independent C++ interface for threading, but to
> date none of the other Boost libraries depend on the use of
> Boost.Threads.  In other words, Boost doesn't require TLS, but it can
> provide TLS ;-)

Yes, this is exactly what I meant.  Mozilla is another good example.
Mozilla does not require TLS, but indeed builds its own API for it - ie,
xpcom does not require it, but does provide it.

While Mozilla therefore has TLS ports for many many platforms, this doesn't
help us directly, as we can't just lift their code (MPL, etc).  But I
believe we could simply lean on them for their implementations at runtime.

> I take it you are planning to provide a way to get the neccessary TLS
> from Python's API (in case it isn't lying about elsewhere), but not
> neccessarily port it to every platform?

I am not sure what you mean by "get the necessary TLS from Python's API".  I
don't see a need for Python to expose any TLS functionality.  If TLS is
required *only* for this thread-state magic, then Python just consumes TLS,
never exposes it.  It obviously does expose an API which internally uses
TLS, but it will not expose TLS itself.  I forsee a "bootstrap prelude
dance" which an extension library must perform, setting up these pointers
exactly once.

The obvious question from this approach is how to deal with *multiple*
libraries in one app.  For example, what happens when a single Python
application wishes to use Boost *and* xpcom, and both attempt their
bootstrap prelude, each providing a TLS implementation?

Off the top of my head, a "first in wins" strategy may be fine - we dont
care *who* provides TLS, so long as we have it.  We don't really have a way
to unload an extension module, so lifetime issues may not get in our way.

Mark.