[Python-ideas] In-process interpreters

[Swapnil Talekar]

On Wed, Nov 17, 2010 at 2:11 PM, Chris Rebert <pyideas at rebertia.com> wrote:

> On Wed, Nov 17, 2010 at 12:26 AM, Swapnil Talekar <swapnil.st at gmail.com>
> wrote:
> > I am working on a project where we have a very peculiar requirements for
> an
> > embedded interpreter. In my opinion these requirements cannot be
> extremely
> > rare. I am posting this message to get some feedback from people who have
> > worked on similar stuff or are interested in such a solution. If such a
> > solution would be useful to a lot people if readily available as open
> > source.
> > We need to run multiple interpreters in-process. Which means we need to
> > create completely independent environments for each interpreter using
> some
> > very dirty hack. Since Python inherently does not support this. Python
> > threads( created using python's threading module ) are not of much help
> > because of the GIL.
> > For this purpose the solution we have adopted is moving all the mutable
> > python globals to heap.
> > Nokia did a similar change to Python while porting it to S60 platform.
> > Reference
> > It seems Nokia has stopped using this solution but I am not aware of how
> are
> > they going about now since I cannot think of any other solution to run
> > multiple in-process interpreters. Jukka, can you throw some light ?
> > Lua already supports this.
> > I would expect other projects like mod_python would also have to face
> > similar problem but I am not aware how they are handling it.
>
> It used* "sub-interpreters" apparently:
> http://modpython.org/live/current/doc-html/pyapi-interps.html
> http://docs.python.org/c-api/init.html#Py_NewInterpreter


Sub-interpreters aren't attractive for the very reasons given in the Bugs
and caveats section of the above link. Moreover Sub-interpreters do not
release memory (as far as I have tested) even after calling
Py_EndInterpreter. This makes them unusable on platforms with high memory
constraint.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101117/34c6680a/attachment.html>