[Python-Dev] Extension modules, Threading, and the GIL

[Gordon McMillan]

On 10 Jan 2003 at 15:00, Thomas Heller wrote:

> "Gordon McMillan" <gmcm@hypernet.com> writes:
[...]
> > Installer freezes in-process COM servers. It does so
> > by using a generic shim dll which gets renamed for
> > each component. Basically, this dll will forward most
> > of the calls on to Mark's PythoncomXX.dll. But it
> > wants to install import hooks.
[...]
> I cannot really believe this. Isn't it the same as
> for normal, unfrozen inprocess COM servers?  

No. COM always loads pythoncom22 in that case.
Note that a Python22 app can load a frozen Python21
COM server just fine.

> The
> shim dll could do the same as pythoncom22.dll does,
> or even rely on it to do the right thing. 

That's what it tries to do. It loads pythoncomXX.dll
and forwards all the calls it can.

> Unfrozen inproc COM works whether the main process is
> Python or not.

Yes, pythoncom doesn't install import hooks.

> > ... so can't
> > install it's import hooks into the right one.
> 
> IMO, it's the frozen DLL rendering the Python
> environment unusable for everything else (the main
> process, for example).

I don't understand that statement at all.

Working with a (same version) Python app is actually
a secondary worry. I'm more bothered that, for
example, Excel can't load 2 frozen servers which
use the same Python.

> I hope using the frozen module mechanism instead of
> import hooks will make this more tolerant. 

But where are those modules frozen? How do they
get installed in the already running Python? What
if mulitple sets of frozen modules (with dups) want
to install themselves?
 
> All this may of course be off-topic for this thread.

It ties into Martin's earlier comments about
threading models. It may be that the solution
lies in using COM's apartment threading, 
instead of free threading. That way, the COM
server could have it's own interpreter state, and
the calls would end up in the right interpreter.
Maybe.

But I don't understand the COM part well enough, 
and Mark's stuff supports free threading, not
apartment threading.  

I really brought all this up to try to widen the
scope from extension modules which can
easily grab an interpreter state and hold
onto it.

-- Gordon
http://www.mcmillan-inc.com/