[issue15292] import hook behavior documentation improvement
New submission from Anders Hammarquist <iko@iko.pp.se>: When testing Eutaxia on PyPy (1.9) I discovered a discrepancy in the path_hooks import hook implementation. In CPython (2.7), if the find_module() method raises ImportError (as imp.find_module() does when it does not find a module in the given path), will cause the search to continue, whereas PyPy would propagate the ImportError. PyPy has now been changed to behave like CPython. The documentation is not entirely clear, but it does not explicitly document the import hook mechanism as eating an ImportError in find_module(). It should probably be made explicit, which ever way it should be. It is not obvious what is the correct behaviour, given the implicit relative imports, where the ImportError simply means that the import hook cannot find the module. Quick testing on CPython 3.3 indicates that it behaves like PyPy did, but as it doesn't do implicit relative imports my test case didn't work as it was. For 3.3, without implicit relative imports, propagating the ImportError feels like the correct behaviour. The attached demonstration needs a file /tmp/test/foo.py that does a top-level import, e.g. "import errno" to demonstrate the discrepancy. ---------- assignee: docs@python components: Documentation files: testimport.py messages: 164998 nosy: docs@python, iko priority: normal severity: normal status: open title: import hook behavior documentation improvement type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file26315/testimport.py _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15292> _______________________________________
Changes by Antoine Pitrou <pitrou@free.fr>: ---------- nosy: +brett.cannon, ncoghlan _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15292> _______________________________________
Nick Coghlan <ncoghlan@gmail.com> added the comment: The PyPy and 3.3 behaviour are actually correct according to the spec, but it's *really* unclear in PEP 302. sys.meta_path accepts finder objects. These are explicitly documented as returning "None" from find_module() to indicate "try the next one" and raising exceptions solely to report problems. However, for reasons that are unknown to me, sys.path_hooks entries (which occupy most of the section on registering hooks) use a different protocol to indicate "try the next one": raising ImportError. Since meta_path and path_hooks are described in the same section, and the meta_path description just says "add finder objects", it's understandable that implementors take the path_hooks protocol description as applying to finders in general :( I would chalk the 2.x (and likely 3.x for x < 3) behaviour up to the only partial implementation of PEP 302 in CPython (until Brett's success in bootstrapping importlib for 3.3). ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15292> _______________________________________
Brett Cannon <brett@python.org> added the comment: Everything Nick said is right: PyPy did it properly according to the spec and CPython 2.7 got it wrong. Unfortunately fixing this now would break code and so it will simply have to stay a Python 2.7 quirk with Python 3.3 and later doing it correctly. So documenting the screw-up would be good so that people know that the solution they use in Python 2.7 won't work in Python 3.3 and later. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15292> _______________________________________
Nick Coghlan <ncoghlan@gmail.com> added the comment: Perhaps the porting section in the 3.3 What's New? ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15292> _______________________________________
Brett Cannon <brett@python.org> added the comment: So I can't reproduce the problem under Python 2.7.3, Python 3.2.3, or a fresh checkout of Python 3.3. This is with both your script, Anders, and writing my own class that defined find_module() to just raise ImportError that I set in either sys.meta_path or in sys.path_importer_cache (both ways let the ImportError propagate). This means I'm not sure exactly what problem you are encountering. Is this on an older version of Python 2.7? Do you have a unit test that can be run which shows the test failing under Python 2.7 but passing under Python 3.3? ---------- status: open -> pending _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15292> _______________________________________
Brett Cannon added the comment: Closing as "works for me" due to lack of reply from OP. ---------- resolution: -> works for me status: pending -> closed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15292> _______________________________________
participants (4)
-
Anders Hammarquist
-
Antoine Pitrou
-
Brett Cannon
-
Nick Coghlan