[issue14657] Avoid two importlib copies

[Eric Snow]

Eric Snow <ericsnowcurrently at gmail.com> added the comment:

Here's my take.  No one will care about _frozen_importlib vs. importlib._bootstrap normally, right?  If __module__/__file__ says _frozen_importlib, it's no big deal.  The only time you care about the distiction for importlib._bootstrap is when you're hacking on _bootstrap.py.  So let's keep the common case in sight and go from there.

There are two sides to the uncommon case:

1. making sure importlib still works after hacking on _bootstrap.py (test_imp, test_import, test_importlib using your new _bootstrap.py).
2. making sure everything still works after hacking on _bootstrap.py (the whole test suite with a new importlib.h?).

For the first part, let's simply ignore the pure Python importlib._bootstrap by default?  Then we stick a context manager in importlib.test.util that enables it.  When you're hacking on _bootstrap.py, you switch it over.  The common path stays pretty clean.

I've attached a patch for the first part which has similarities to Antoine's.  (I didn't apply the context manager to the importlib test cases though.)

For that second part, something along the lines of what Nick has posted would be pretty close, but I'm not sure it's worth it.  From what I understand, Nick's patch would add yet another import (importlib) to startup to cover a situation that happens very infrequently (hacking _bootstrap.py).  However, I'm torn because...

...dealing with a busted importlib.h is not fun.  Is there a different approach we could take for that second part?  Perhaps something like this:

1. python starts up normally.
2. we clear out all the entire import state except for builtins.
3. we stick importlib._bootstrap in place.
4. we set builtins.__import__ to importlib.__import__.
5. we re-populate sys.modules by reloading all the modules that were in there before (?).
6. we run the test suite against this new import state.
7. ...
8. profit!

I'm probably missing something here, but I expect we could stick something like that in some place like importlib.test.util.  Would that be sufficient to mitigate the chance of breaking importlib.h?


------------------------------------

Example of using my patch:

>>> import sys
>>> import importlib.test.util as util
>>> importlib._bootstrap
<module '_frozen_importlib' from '<frozen>'>
>>> sys.modules['importlib._bootstrap']
<module '_frozen_importlib' from '<frozen>'>
>>> with util.bootstrap_context(importlib, importlib._pure_bootstrap):
...     importlib._bootstrap
...     sys.modules['importlib._bootstrap']
...
<module 'importlib._bootstrap' from '/home/esnow/projects/cpython/Lib/importlib/_bootstrap.py'>
<module 'importlib._bootstrap' from '/home/esnow/projects/cpython/Lib/importlib/_bootstrap.py'>
>>> importlib._bootstrap
<module '_frozen_importlib' from '<frozen>'>
>>> sys.modules['importlib._bootstrap']
<module '_frozen_importlib' from '<frozen>'>

----------
Added file: http://bugs.python.org/file25479/issue14657_safe_bootstrap.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14657>
_______________________________________