Initialization hook for extenders
I work on several projects that have initialization requirements that need to grab control after Py_Initialize(), but before any user code runs (via input, script, -c, etc...).
Note that these are Python clones that take advantage of an installed python (using its $prefix/lib/pythonx.x/*.py and site-packages/*)
We could use
PyImport_AppendInittab("sitecustomize",initsitecustomize);
But if there already IS customization in sitecustomize.py, I've blown it away (and have to look it up and force an import). And if someone uses the -S flag, I'm screwed.
I propose a hook styled after Py_AtExit(func) called Py_AtInit(func) which maintains a list of functions that are called in Py_Initialize right after main and site initializations.
If the hook isn't used, then the cost is a single extra function call at initialization. Here's a spurious example: A customer wants a version of python that has all the math functions and his extensions to act like builtins...
I would write (without refcnt or error checks ;-):
#include "Python.h" static void after_init(void) { PyObject *builtin,*builtin_dict,*math,*math_dict,*user,*user_dict;
builtin = PyImport_ImportModule("__builtin__"); builtin_dict = PyModule_GetDict(builtin); math = PyImport_ImportModule("math"); math_dict = PyModule_GetDict(math); user = PyImport_ImportModule("user"); user_dict = PyModule_GetDict(math);
PyDict_Update(builtin_dictionary, math_dict); PyDict_Update(builtin_dictionary, user_dict); }
int main(int argc, char** argv) { PyImport_AppendInittab("user",inituser); Py_AtInit(after_init);
return Py_Main(argc, argv); }
voila! An extended Python with new builtins.
I actually want this to do some MPI initialization to setup a single user prompt with broadcast which has to run after Py_Initialize() but before the import of readline.
I've attached a copy of the patch (also going to patches at sf.net)
Pat
"Patrick J. Miller" patmiller@llnl.gov writes:
I actually want this to do some MPI initialization to setup a single user prompt with broadcast which has to run after Py_Initialize() but before the import of readline.
-1. It is easy enough to copy the code of Py_Main, and customize it for special requirements. The next user may want to have a hook to put additional command line options into Py_Main, YAGNI.
Regards, Martin
"Martin v. Löwis" wrote:
-1. It is easy enough to copy the code of Py_Main, and customize it for special requirements. The next user may want to have a hook to put additional command line options into Py_Main, YAGNI.
It's not easy.
Not if you simply want to link against an installed Python.
Nor so if you want to build against 2.1 2.2 and 2.3 ... libraries.
There are subtle changes that bite you in the ass if you don't physically copy the right source forward.
We did copy forward main.c, but found that every time we updated Python, we had to "rehack" main to make sure we had all the options and flags and initialization straight.
I think the hook is extremely cheap, very short, looks almost exactly like Py_AtExit() and solves the problem directly.
Pat
Patrick J. Miller wrote:
It's not easy.
Not if you simply want to link against an installed Python.
Why not? Just don't call the function Py_Main.
Nor so if you want to build against 2.1 2.2 and 2.3 ... libraries.
Again, I can't see a reason why that is.
There are subtle changes that bite you in the ass if you don't physically copy the right source forward.
For example?
We did copy forward main.c, but found that every time we updated Python, we had to "rehack" main to make sure we had all the options and flags and initialization straight.
That is not necessary. What would be the problem if you just left your function as it was in Python 2.1?
I think the hook is extremely cheap, very short, looks almost exactly like Py_AtExit() and solves the problem directly.
Unfortunately, the problem is one that almost nobody ever has, and supporting that API adds a maintenance burden. It is better if the maintenance burden is on your side than on the Python core.
If you think you really need this, write a PEP, ask the community, and wait for BDFL pronouncement. I'm still -1.
Regards, Martin
Martin,
Sorry you disagree. I think that the issue is still important and other pieces of the API are already in this direction.
For instance, there is no need to have PyImport_AppendInittab because you can hack config.c (which you can get from $prefix/lib/pythonx.x/config/config.c) and in fact many people did exactly that but it made for a messy extension until the API call made it clean and direct.
You don't need Py_AtExit() because you can call through to atexit.register() to put the function in.
The list goes on...
I still think that Py_AtInit() is clean, symmetric with Py_AtExit(), and solves a big problem for extenders who wish to address localization from within C (as opposed to sitecustomize.py).
This is a 10 line patch with 0 runtime impact that requires no maintanence to move forward with new versions. If it were more than that, I could better understand your objections.
Hope that I can get you to at least vote 0 instead of -1.
Cheers,
Pat
participants (3)
-
"Martin v. Löwis" -
martin＠v.loewis.de
-
Patrick J. Miller