[Python-Dev] multiple interpreters and extension modules

[Jeremy Kloth]

On Friday 22 December 2006 5:02 pm, Josiah Carlson wrote:
> Jeremy Kloth <jeremy.kloth at 4suite.org> wrote:
> > [[ This may be somewhat c.l.p.-ish but I feel that this crossed into
> > CPython development enough to merit posting here ]]
> >
> > I have received a bug report for 4Suite that involves a
> > PyObject_IsInstance check failing for what appears to be the correct
> > class, that is, the class names match.  With some investigating, I have
> > found that the true problem is with multiple interpreters.  The reason
> > for this is that each sub-interpreter has a "new" copy of any pure Python
> > module. The following issues are also true for modules that have been
> > reloaded, but I think that is common knowledge.  I mention it only for
> > completeness.
>
> If I remember correctly, Python allows you to use multiple interpreters
> in the same process, but it makes no guarantees as to their correctness
> when running.
>
> See this post for further discussion on the issue:
> http://mail.python.org/pipermail/python-list/2004-January/244343.html
>
> You can also search for 'multiple python interpreters single process' in
> google without quotes to hear people lament over the (generally broken)
> multiple Python interpreter support.

The problem here is that it is mod_python using the multiple interpreters.  We 
have no control over that.  What I'm proposing is fixing the extension module 
support for multiple interpreters with the bonus of adding extension module 
finalization which I've seen brought up here before.

Fixing this does require support by the extension module author, but if that 
author doesn't feel the need to work in mod_python (if, of course, they load 
module level constants), that is their loss.

Is 4Suite that different in its use of hybrid Python and C extensions?  There 
is lots of back and forth between the two layers and performance is critical.  
I really don't feel like recoding thousands of lines of Python code into C 
just to get 4Suite to work in mod_python without error.

-- 
Jeremy Kloth
http://4suite.org/