[issue23773] importlib does not properly remove frames when invoking external import hooks

[Shiz]

New submission from Shiz:

When adding a custom module loader to sys.meta_path, importlib does not properly remove its frames before invoking it. This results in a weird traceback with importlib._bootstrap frames in if an error occurs during load_module(), like such:

Traceback (most recent call last):
  File "/Users/mark/Development/Projects/rave/rave/bootstrap/__init__.py", line 102, in bootstrap_game
    __import__(MODULE_PACKAGE + '.' + mod.replace('.py', ''))
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/__init__.py", line 1, in <module>
    from . import common, core3
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/__init__.py", line 11, in <module>
    from .texture import Texture, Image
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/texture.py", line 8, in <module>
    raise ValueError

Attached is a patch against the current hg head that makes sure module loaders are invoked through _call_with_frames_removed, so that the output becomes a lot more readable:

Traceback (most recent call last):
  File "/Users/mark/Development/Projects/rave/rave/bootstrap/__init__.py", line 102, in bootstrap_game
    __import__(MODULE_PACKAGE + '.' + mod.replace('.py', ''))
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/__init__.py", line 1, in <module>
    from . import common, core3
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/__init__.py", line 11, in <module>
    from .texture import Texture, Image
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/texture.py", line 8, in <module>
    raise ValueError
ValueError

Ideally it would remove the calls /within/ the module loader itself too, but I do not see an easy way to do this without changing the entire _call_with_frames_removed semantics. This fixes most of the issues, though.

----------
components: Library (Lib)
files: cpython-3ac58de829ef-fix-external-module-loader-call-traceback.patch
keywords: patch
messages: 239219
nosy: shiz
priority: normal
severity: normal
status: open
title: importlib does not properly remove frames when invoking external import hooks
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file38681/cpython-3ac58de829ef-fix-external-module-loader-call-traceback.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23773>
_______________________________________