Strange import behaviour, recently introduced
I'm running into strange problems with import in frozen Mac programs. On the Mac a program is frozen in a rather different way from how it happens on Unix/Windows: basically all .pyc files are stuffed into resources, and if the import code comes across a file on sys.path it will look for PYC resources in that file. So, you freeze a program by stuffing all your modules into the interpreter executable as PYC resources and setting sys.path to contain only the executable file, basically. This week I noticed that these resource imports have suddenly become very very slow. Whereas startup time of my application used to be around 2 seconds (where the non-frozen version took 6 seconds) it now takes almost 20 times as long. The non-frozen version still takes 6 seconds. I suspect this may have something to do with recent mods to the import code, but attempts to pinpoint the problem have failed so far (somehow the profiler crashes my app). I've put a breakpoint at import.c:check_case(), and it isn't hit (as is to be expected), so that isn't the problem. Does anyone have a hint for where I could start looking? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++
Jack Jansen wrote:
I'm running into strange problems with import in frozen Mac programs.
On the Mac a program is frozen in a rather different way from how it happens on Unix/Windows: basically all .pyc files are stuffed into resources, and if the import code comes across a file on sys.path it will look for PYC resources in that file. So, you freeze a program by stuffing all your modules into the interpreter executable as PYC resources and setting sys.path to contain only the executable file, basically.
This week I noticed that these resource imports have suddenly become very very slow. Whereas startup time of my application used to be around 2 seconds (where the non-frozen version took 6 seconds) it now takes almost 20 times as long. The non-frozen version still takes 6 seconds.
The most recent version calls PyImport_ImportModuleEx() for '__builtin__' for every import of __builtin__ without caching the result in a static variable. Can this be the cause? Thomas Heller
Jack Jansen wrote:
This week I noticed that these resource imports have suddenly become very very slow. Whereas startup time of my application used to be around 2 seconds (where the non-frozen version took 6 seconds) it now takes almost 20 times as long. The non-frozen version still takes 6 seconds.
[Thomas Heller]
The most recent version calls PyImport_ImportModuleEx() for '__builtin__' for every import of __builtin__ without caching the result in a static variable.
Can this be the cause?
Would this help? *** import.c 2001/02/20 21:43:24 2.162 --- import.c 2001/02/22 02:24:55 *************** *** 1873,1878 **** --- 1873,1879 ---- { static PyObject *silly_list = NULL; static PyObject *builtins_str = NULL; + static PyObject *builtin_str = NULL; static PyObject *import_str = NULL; PyObject *globals = NULL; PyObject *import = NULL; *************** *** 1887,1892 **** --- 1888,1896 ---- builtins_str = PyString_InternFromString("__builtins__"); if (builtins_str == NULL) return NULL; + builtin_str = PyString_InternFromString("__builtin__"); + if (builtin_str == NULL) + return NULL; silly_list = Py_BuildValue("[s]", "__doc__"); if (silly_list == NULL) return NULL; *************** *** 1902,1913 **** } else { /* No globals -- use standard builtins, and fake globals */ PyErr_Clear(); ! builtins = PyImport_ImportModuleEx("__builtin__", ! NULL, NULL, NULL); if (builtins == NULL) return NULL; globals = Py_BuildValue("{OO}", builtins_str, builtins); if (globals == NULL) goto err; --- 1906,1918 ---- } else { /* No globals -- use standard builtins, and fake globals */ + PyInterpreterState *interp = PyThreadState_Get()->interp; PyErr_Clear(); ! builtins = PyDict_GetItem(interp->modules, builtin_str); if (builtins == NULL) return NULL; + Py_INCREF(builtins); globals = Py_BuildValue("{OO}", builtins_str, builtins); if (globals == NULL) goto err; --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (3)
-
Guido van Rossum
-
Jack Jansen
-
Thomas Heller