[issue21617] importlib reload can fail with AttributeError if module removed from sys.path

[Ned Deily]

New submission from Ned Deily:

There are significant differences in behavior between Python 2.7, 3.3, and 3.4 (or current default) when using import reload() while manipulating sys.path.  These differences cause unexpected behavior in the "Run Script" command of the TextMate 2 editor's Python bundle.  TM2's Python "Run Script" is designed to work with either Python 2 or 2 interpreters.  To do things like capture script stdout and stderr, the bundle launches a Python interpreter while prepending a directory with a custom sitecustomize module to sys.path.  The TM2 sitecustomize then removes its directory from sys.path and uses reload to attempt to import any existing sitecustomize module.  It then proceeds to do its TM2 customizations.  In simplified pseudo-code:

# TM_DIRECTORY/sitecustomize.py
import sys
# remove this copy of sitecustomize from import path
if "TM_DIRECTORY" in sys.path:
    sys.path.remove("TM_DIRECTORY")
try:
    import sitecustomize    # get module reference
    if sys.version_info[0] >= 3:
        from imp import reload
    reload(sitecustomize)   # try to import another
                            # using altered path
except ImportError:
    pass                    # no other sitecustomize found
# do TM2 customizations here
# ...

this seems to work just fine with Python 2.7.  With Python 3.3 the reload causes the originally imported sitecustomize to be used again, regardless of whether another sitecustomize exists on sys.path.  For Python 3.4, 1d67eb1df5a9 for Issue19851 changed reloading.  Now for 3.4 (and default), if another sitecustomize exists on sys.path, the behavior is as expected as in 2.7: the other sitecustomize is imported by reload and executes.  However, if there is no other sitecustomize, 3.4/default now gets an AttributeError exception in reload, rather than the expected ImportError.  Enabling PYTHONVERBOSE shows:

  File "./lib/python3.5/importlib/__init__.py", line 148, in reload
    _bootstrap._exec(spec, module)
  File "<frozen importlib._bootstrap>", line 1083, in _exec
AttributeError: 'NoneType' object has no attribute 'name'

And this is due to the call in reload() to _bootstrap._find_spec() returning None (Lib/importlib/__init__.py:147), presumably since there no longer is a sitecustomize.
It looks like the following patch fixes the problem for this case:

diff --git a/Lib/importlib/__init__.py b/Lib/importlib/__init__.py
--- a/Lib/importlib/__init__.py
+++ b/Lib/importlib/__init__.py
@@ -145,6 +145,9 @@
             pkgpath = None
         target = module
         spec = module.__spec__ = _bootstrap._find_spec(name, pkgpath, target)
+        if not spec:
+            msg = "module {} cannot be reloaded"
+            raise ImportError(msg.format(name), name=name)
         _bootstrap._exec(spec, module)
         # The module may have replaced itself in sys.modules!
         return sys.modules[name]

I'm not sure if this is a correct or sufficient change in the general case.

http://stackoverflow.com/questions/23962319/error-output-in-textmate-2-using-python-3-4

----------
files: test_sitecustomize.sh
messages: 219436
nosy: brett.cannon, eric.snow, ned.deily
priority: normal
severity: normal
status: open
title: importlib reload can fail with AttributeError if module removed from sys.path
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file35416/test_sitecustomize.sh

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21617>
_______________________________________