2.6, 3.0, and truly independent intepreters

Andy O'Meara andy55 at gmail.com
Sat Oct 25 17:26:18 EDT 2008


> Andy O'Meara wrote:
> > I would definitely agree if there was a context (i.e. environment)
> > object passed around then perhaps we'd have the best of all worlds.
>
> Moreover, I think this is probably the *only* way that
> totally independent interpreters could be realized.
>
> Converting the whole C API to use this strategy would be
> a very big project. Also, on the face of it, it seems like
> it would render all existing C extension code obsolete,
> although it might be possible to do something clever with
> macros to create a compatibility layer.
>
> Another thing to consider is that passing all these extra
> pointers around everywhere is bound to have some effect
> on performance.


Good points--I would agree with you on all counts there.  On the
"passing a context everywhere" performance hit, perhaps one idea is
that all objects could have an additional field that would point back
to their parent context (ie. their interpreter).  So the only
prototypes that would have to be modified to contain the context ptr
would be the ones that inherently don't take any objects. This would
conveniently and generally correspond to procs associated with
interpreter control (e.g. importing modules, shutting down modules,
etc).


> Andy O'Meara wrote:
> > - each worker thread makes its own interpreter, pops scripts off a
> > work queue, and manages exporting (and then importing) result data to
> > other parts of the app.
>
> I hope you realize that starting up one of these interpreters
> is going to be fairly expensive.

Absolutely.  I had just left that issue out in an effort to keep the
discussion pointed, but it's a great point to raise.  My response is
that, like any 3rd party industry package, I'd say this is the
expectation (that context startup and shutdown is non-trivial and to
should be minimized for performance reasons).  For simplicity, my
examples didn't talk about this issue but in practice, it'd be typical
for apps to have their "worker" interpreters persist as they chew
through jobs.


Andy





More information about the Python-list mailing list