Making the __import__ hook available early...
There has been some discussion on the import-sig about using the __import__ hook for practically all imports, even early in the startup phase. This allows import hooks to completely take over the import mechanism even for the Python standard lib. Thomas Heller has provided a patch which I am currently checking. Basically all C level imports using PyImport_ImportModule() are then redirected to PyImport_Import() which uses the __import__ hook if available. My testing has so far not produced any strange effects. If anyone objects to this change, please speak up. Else, I'll check it in later today. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
There has been some discussion on the import-sig about using the __import__ hook for practically all imports, even early in the startup phase. This allows import hooks to completely take over the import mechanism even for the Python standard lib.
Thomas Heller has provided a patch which I am currently checking. Basically all C level imports using PyImport_ImportModule() are then redirected to PyImport_Import() which uses the __import__ hook if available.
My testing has so far not produced any strange effects. If anyone objects to this change, please speak up. Else, I'll check it in later today.
One remaining difference I noted between running 'rt.bat -d' with the CVS version and the patched version is that the former reported [56931 refs] and the latter [56923 refs]. Thomas
Thomas Heller wrote:
There has been some discussion on the import-sig about using the __import__ hook for practically all imports, even early in the startup phase. This allows import hooks to completely take over the import mechanism even for the Python standard lib.
Thomas Heller has provided a patch which I am currently checking. Basically all C level imports using PyImport_ImportModule() are then redirected to PyImport_Import() which uses the __import__ hook if available.
My testing has so far not produced any strange effects. If anyone objects to this change, please speak up. Else, I'll check it in later today.
One remaining difference I noted between running 'rt.bat -d' with the CVS version and the patched version is that the former reported [56931 refs] and the latter [56923 refs].
This is probably due to the interning of strings; nothing to worry about, I guess. The patch implements the same refcounting as before the patch, so it is clearly not the cause of the different figures. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
[M.-A. Lemburg]
There has been some discussion on the import-sig about using the __import__ hook for practically all imports, even early in the startup phase. This allows import hooks to completely take over the import mechanism even for the Python standard lib.
Thomas Heller has provided a patch which I am currently checking. Basically all C level imports using PyImport_ImportModule() are then redirected to PyImport_Import() which uses the __import__ hook if available.
My testing has so far not produced any strange effects. If anyone objects to this change, please speak up. Else, I'll check it in later today.
I don't understand the change, from the above. Neither exactly what it does nor why it's being done. So, impossible to say. Was the patch posted to SourceForge? Does it have a bad effect on startup time? Is there any *conceivable* way in which it could change semantics? Or, if not, what's the point?
Tim Peters wrote:
[M.-A. Lemburg]
There has been some discussion on the import-sig about using the __import__ hook for practically all imports, even early in the startup phase. This allows import hooks to completely take over the import mechanism even for the Python standard lib.
Thomas Heller has provided a patch which I am currently checking. Basically all C level imports using PyImport_ImportModule() are then redirected to PyImport_Import() which uses the __import__ hook if available.
My testing has so far not produced any strange effects. If anyone objects to this change, please speak up. Else, I'll check it in later today.
I don't understand the change, from the above. Neither exactly what it does nor why it's being done. So, impossible to say. Was the patch posted to SourceForge? Does it have a bad effect on startup time? Is there any *conceivable* way in which it could change semantics? Or, if not, what's the point?
I've already checked it in, but for completeness ;-) ... The problem was that tools like Thomas Heller's pyexe, Gordon's installer and other similar tools which try to pack Python byte code into a single archive need to provide an import hook which then redirects imports to the archive. This was already well possible for third-party code, but some of the standard modules in the Python lib used PyImport_ImportModule() directly to import modules and this prevented the inclusion of the referenced modules in the archive. When no import hook is in place, the patch does not have any effect -- semantics are the same as before. Import performance for those few cases where PyImport_ImportModule() was used will be a tad slower, but probably negligable due to the overhead caused by the file IO. With the hook in place, the patch now properly redirects these low-level imports to the __import__ hook. Semantics will then be those which the __import__ hook defines. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
participants (3)
-
M.-A. Lemburg -
Thomas Heller -
Tim Peters