[Distutils] Installing namespace packages with pip inserts strange modules into sys.modules

[Thomas Heller]

While trying to add support for PEP420 implicit namespace packages
support to py2exe's modulefinder, I noticed something strange.

I'm using the wheezy.template and zope.interface packages for my tests,
they are installed with pip.

pip creates wheezy.template-0.1.155-py3.4-nspkg.pth and
zope.interface-4.1.1-py3.4-nspkg.pth files that are not present in the 
package sources.  The contents is like this:

"""
import sys,types,os; p = 
os.path.join(sys._getframe(1).f_locals['sitedir'], *('wheezy',)); ie = 
os.path.exists(os.path.join(p,'__init__.py')); m = not ie and 
sys.modules.setdefault('wheezy',types.ModuleType('wheezy')); mp = (m or 
[]) and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p)
"""

What is the purpose of these files?  (The packages seem to work also
when I delete these files; which is what I expected from implicit
namespace packages...)


These .pth-files have the effect that they insert the modules 'wheezy'
and 'zope' into sys.modules, even when I don't import anything
from the packages.  However, these modules are IMO damaged, they
only have __doc__, __name__, and __path__ attributes (plus __loader__
and __spec__ in Python 3.4, both set to None).

Reloading the package in Python 3.3 raises an error, and
importlib.find_loader("zope") also:

 >>> sys.modules["zope"]
<module 'zope'>
 >>> import zope
 >>> zope
<module 'zope'>
 >>>
 >>> imp.reload(zope)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "C:\Python33\lib\imp.py", line 276, in reload
     module.__loader__.load_module(name)
AttributeError: 'module' object has no attribute '__loader__'
 >>>
 >>> importlib.find_loader("zope", None)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "C:\Python33\lib\importlib\__init__.py", line 64, in find_loader
     loader = sys.modules[name].__loader__
AttributeError: 'module' object has no attribute '__loader__'
 >>>

Trying the same in Python 3.4, behaviour is quite similar although
imp.reload() 'fixes' that package:

 >>> import sys
 >>> sys.modules["zope"]
<module 'zope'>
 >>> dir(sys.modules["zope"])
['__doc__', '__loader__', '__name__', '__package__', '__path__', '__spec__']
 >>> sys.modules["zope"].__spec__
 >>>
 >>> import zope
 >>> zope
<module 'zope'>
 >>> import importlib.util
 >>> importlib.util.find_spec("zope", None)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "C:\Python34\lib\importlib\util.py", line 100, in find_spec
     raise ValueError('{}.__spec__ is None'.format(name))
ValueError: zope.__spec__ is None
 >>> import imp
 >>> imp.reload(zope)
<module 'zope' (namespace)>
 >>> importlib.util.find_spec("zope", None)
ModuleSpec(name='zope', loader=None, origin='namespace', 
submodule_search_locations=_NamespacePath(['C:\\Python34\\lib\\site-packages\\zope']))
 >>>


Thanks for any insights,
Thomas