Initialization hook for extenders
I work on several projects that have initialization requirements that
need to grab control after Py_Initialize(), but before any user code
runs (via input, script, -c, etc...).
Note that these are Python clones that take advantage of an installed
python (using its $prefix/lib/pythonx.x/*.py and site-packages/*)
We could use
PyImport_AppendInittab("sitecustomize",initsitecustomize);
But if there already IS customization in sitecustomize.py, I've
blown it away (and have to look it up and force an import).
And if someone uses the -S flag, I'm screwed.
I propose a hook styled after Py_AtExit(func) called Py_AtInit(func)
which maintains a list of functions that are called in Py_Initialize
right after main and site initializations.
If the hook isn't used, then the cost is a single extra function
call at initialization. Here's a spurious example: A customer wants
a version of python that has all the math functions and his
extensions to act like builtins...
I would write (without refcnt or error checks ;-):
#include "Python.h"
static void after_init(void) {
PyObject *builtin,*builtin_dict,*math,*math_dict,*user,*user_dict;
builtin = PyImport_ImportModule("__builtin__");
builtin_dict = PyModule_GetDict(builtin);
math = PyImport_ImportModule("math");
math_dict = PyModule_GetDict(math);
user = PyImport_ImportModule("user");
user_dict = PyModule_GetDict(math);
PyDict_Update(builtin_dictionary, math_dict);
PyDict_Update(builtin_dictionary, user_dict);
}
int main(int argc, char** argv) {
PyImport_AppendInittab("user",inituser);
Py_AtInit(after_init);
return Py_Main(argc, argv);
}
voila! An extended Python with new builtins.
I actually want this to do some MPI initialization to setup a
single user prompt with broadcast which has to run after
Py_Initialize() but before the import of readline.
I've attached a copy of the patch (also going to patches
at sf.net)
Pat
--
Patrick Miller | (925) 423-0309 |
http://www.llnl.gov/CASC/people/pmiller
Son, when you grow up you will know who I really am.
I am just a child like you who has been forced to act
responsibly. -- Rod Byrnes
Index: dist/src/Include/pythonrun.h
===================================================================
RCS file: /cvsroot/python/python/dist/src/Include/pythonrun.h,v
retrieving revision 2.62
diff -c -r2.62 pythonrun.h
*** dist/src/Include/pythonrun.h 13 Feb 2003 22:07:52 -0000 2.62
--- dist/src/Include/pythonrun.h 30 Apr 2003 22:04:13 -0000
***************
*** 75,80 ****
--- 75,81 ----
PyAPI_FUNC(void) PyErr_Display(PyObject *, PyObject *, PyObject *);
PyAPI_FUNC(int) Py_AtExit(void (*func)(void));
+ PyAPI_FUNC(int) Py_AtInit(void (*func)(void));
PyAPI_FUNC(void) Py_Exit(int);
Index: dist/src/Python/pythonrun.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/pythonrun.c,v
retrieving revision 2.193
diff -c -r2.193 pythonrun.c
*** dist/src/Python/pythonrun.c 22 Apr 2003 11:18:00 -0000 2.193
--- dist/src/Python/pythonrun.c 30 Apr 2003 22:04:16 -0000
***************
*** 106,111 ****
--- 106,135 ----
return flag;
}
+ #define NINITFUNCS 32
+ static void (*initfuncs[NINITFUNCS])(void);
+ static int ninitfuncs = 0;
+
+ int Py_AtInit(void (*func)(void))
+ {
+ if (ninitfuncs >= NINITFUNCS)
+ return -1;
+ if (!func)
+ return -1;
+ initfuncs[ninitfuncs++] = func;
+ return 0;
+ }
+
+ static void initinitialize(void)
+ {
+ int i;
+ for(i=0;i
"Patrick J. Miller"
I actually want this to do some MPI initialization to setup a single user prompt with broadcast which has to run after Py_Initialize() but before the import of readline.
-1. It is easy enough to copy the code of Py_Main, and customize it for special requirements. The next user may want to have a hook to put additional command line options into Py_Main, YAGNI. Regards, Martin
"Martin v. Löwis" wrote:
-1. It is easy enough to copy the code of Py_Main, and customize it for special requirements. The next user may want to have a hook to put additional command line options into Py_Main, YAGNI.
It's not easy. Not if you simply want to link against an installed Python. Nor so if you want to build against 2.1 2.2 and 2.3 ... libraries. There are subtle changes that bite you in the ass if you don't physically copy the right source forward. We did copy forward main.c, but found that every time we updated Python, we had to "rehack" main to make sure we had all the options and flags and initialization straight. I think the hook is extremely cheap, very short, looks almost exactly like Py_AtExit() and solves the problem directly. Pat -- Patrick Miller | (925) 423-0309 | http://www.llnl.gov/CASC/people/pmiller You can discover more about a person in an hour of play than in a year of discussion. -- Plato, philosopher (427-347 BCE)
Patrick J. Miller wrote:
It's not easy.
Not if you simply want to link against an installed Python.
Why not? Just don't call the function Py_Main.
Nor so if you want to build against 2.1 2.2 and 2.3 ... libraries.
Again, I can't see a reason why that is.
There are subtle changes that bite you in the ass if you don't physically copy the right source forward.
For example?
We did copy forward main.c, but found that every time we updated Python, we had to "rehack" main to make sure we had all the options and flags and initialization straight.
That is not necessary. What would be the problem if you just left your function as it was in Python 2.1?
I think the hook is extremely cheap, very short, looks almost exactly like Py_AtExit() and solves the problem directly.
Unfortunately, the problem is one that almost nobody ever has, and supporting that API adds a maintenance burden. It is better if the maintenance burden is on your side than on the Python core. If you think you really need this, write a PEP, ask the community, and wait for BDFL pronouncement. I'm still -1. Regards, Martin
Martin, Sorry you disagree. I think that the issue is still important and other pieces of the API are already in this direction. For instance, there is no need to have PyImport_AppendInittab because you can hack config.c (which you can get from $prefix/lib/pythonx.x/config/config.c) and in fact many people did exactly that but it made for a messy extension until the API call made it clean and direct. You don't need Py_AtExit() because you can call through to atexit.register() to put the function in. The list goes on... I still think that Py_AtInit() is clean, symmetric with Py_AtExit(), and solves a big problem for extenders who wish to address localization from within C (as opposed to sitecustomize.py). This is a 10 line patch with 0 runtime impact that requires no maintanence to move forward with new versions. If it were more than that, I could better understand your objections. Hope that I can get you to at least vote 0 instead of -1. Cheers, Pat -- Patrick Miller | (925) 423-0309 | http://www.llnl.gov/CASC/people/pmiller You can never solve a problem on the level on which it was created. -- Albert Einstein, physicist, Nobel laureate (1879-1955)
participants (3)
-
"Martin v. Löwis"
-
martin@v.loewis.de
-
Patrick J. Miller