questions on dynamically binding the Python interpreter

[has]

Hi all, need a little bit of advice on dynamically binding an embedded
Python interpreter.

First, the code for anyone that wants a look:

http://trac.macosforge.org/projects/appscript/browser/py-osacomponent/trunk/PyOSA

It's a Python OSA component for OS X. The idea is that you wrap up a
scripting language interpreter as a Carbon Component Manager component
(similar to writing a COM component); applications can then call the
OSA API to load this component into memory whenever they need to run
scripts written in that language. The advantage of this approach over
embedding an interpreter directly in the application is that it's
language-agnostic, so application users can write their scripts in any
language they like (AppleScript, JavaScript, Python, etc.), as long as
there's an OSA component available for it (AppleScript, JavaScriptOSA,
PyOSA, etc.). Great concept - at least in theory.

Now, one of the things about the OSA is that, as one of the old MacOS
school, it was pretty much designed on the assumption that
interpreters would be hosted by the application process itself.
Whereas Python is more of the Unix school, and its own preference
would no doubt be to run as separate processes from the main
application process, with two-way IPC to hook the two together. I'll
likely attempt the latter approach later on (which'll no doubt open a
whole new can of worms), but for now I'm having a go at the former as
it's appears a bit simpler to implement (at least on paper) and will
have better runtime performance (since there's no IPC overhead).

For flexibility, and compatibility with application processes that may
already a Python framework loaded, I'm linking Python dynamically via
CoreFoundation's CFBundle API (which basically provides a nice wrapper
around dlopen &co., amongst other things). I've made some reasonable
progress so far, and the pythonloader code can now locate and bind a
Python.framework without triggering bus errors or anything (a big
achievement for me;). However, there's a couple of places I'm a bit
uncertain on how to proceed:

1. Py_IncRef() and Py_DecRef() only appear to have been added in
Python 2.4; is this right? I really need to support Python 2.3 as well
(since that's the version included as standard on OS X 10.3 & 10.4).
At the moment when 2.3 is imported I'm providing it stubs of these
functions that don't really do anything except leak like sieves;
obviously I'd like to replace these. Do I need to start copying and
pasting chunks of header/code out of Python 2.3, or is there a better
way of doing things?

2. I need to make several C functions callable from within the Python
code. Actually these are part of a larger ADT - the opaque data being
passed in separately as a CObject - but implementing them as a Python
type was going to be more work so the ADT approach seemed like a good
idea at the time. At the moment I'm using PyMethodDef +
Py_InitModule4() to wrap those C functions as an extension and inject
it into the interpreter's module namespace; the problem I've found is
that this API is version-sensitive (e.g. 1012 in older Python vs 1013
in newer). Is there some other way to expose those C function to
Python that isn't version-sensitive? If not, what's the best way to
work with the existing API so that users don't get inflicted with
warnings or worse?


Lastly, one other question not directly related to the above: I would
like to provide a degree of insulation between unrelated scripts
(separate module namespaces in particular); and while I realise they
still provide than perfect separation, sub-interpreters do seem to be
the one and only game in town as far as in-process solutions go. Just
how badly am I going to smoke things when I try to enable the sub-
interpreter support? Should I take out house insurance, move well
clear of inhabited areas, etc? Anywhere I can get more information on
the techniques and pitfalls involved? (The Python documentation is
terribly thin on this stuff.)

Many thanks, and apologies for length,

has