On Fri, Sep 12, 2014 at 8:38 AM, Thomas Heller wrote:

While trying to add support for PEP420 implicit namespace packages support to py2exe's modulefinder, I noticed something strange.

I'm using the wheezy.template and zope.interface packages for my tests, they are installed with pip.

pip creates wheezy.template-0.1.155-py3.4-nspkg.pth and zope.interface-4.1.1-py3.4-nspkg.pth files that are not present in the package sources. The contents is like this:

""" import sys,types,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('wheezy',)); ie = os.path.exists(os.path.join(p,'__init__.py')); m = not ie and sys.modules.setdefault('wheezy',types.ModuleType('wheezy')); mp = (m or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p) """

What is the purpose of these files? (The packages seem to work also when I delete these files; which is what I expected from implicit namespace packages...)

These .pth-files have the effect that they insert the modules 'wheezy' and 'zope' into sys.modules, even when I don't import anything from the packages. However, these modules are IMO damaged, they only have __doc__, __name__, and __path__ attributes (plus __loader__ and __spec__ in Python 3.4, both set to None).

Reloading the package in Python 3.3 raises an error, and importlib.find_loader("zope") also:

...

...

...

sys.modules["zope"] import zope zope

imp.reload(zope) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Python33\lib\imp.py", line 276, in reload module.__loader__.load_module(name) AttributeError: 'module' object has no attribute '__loader__'

importlib.find_loader("zope", None) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Python33\lib\importlib\__init__.py", line 64, in find_loader loader = sys.modules[name].__loader__ AttributeError: 'module' object has no attribute '__loader__'

Trying the same in Python 3.4, behaviour is quite similar although imp.reload() 'fixes' that package:

...

...

...

import sys sys.modules["zope"] dir(sys.modules["zope"]) ['__doc__', '__loader__', '__name__', '__package__', '__path__', '__spec__'] sys.modules["zope"].__spec__

import zope zope import importlib.util importlib.util.find_spec("zope", None) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Python34\lib\importlib\util.py", line 100, in find_spec raise ValueError('{}.__spec__ is None'.format(name)) ValueError: zope.__spec__ is None import imp imp.reload(zope) importlib.util.find_spec("zope", None) ModuleSpec(name='zope', loader=None, origin='namespace', submodule_search_locations=_NamespacePath(['C:\\Python34\\lib\\site-packages\\zope']))

Hi Thomas, I've dealt with this issue myself extensively in the past, but it's been a while so I might have to give it a bit of thought before I comment more on detail (if no one else does first). But I don't think (unless there's something new I don't know about) those come directly from pip. Rather, they are created by setuptools. The issue here is that before Python had any built-in notion of namespace packages, namespace packages were a feature in setuptools, and this was the hack, I suppose, that allowed it to work. Unfortunately as different versions of Python have grown different levels of support for namespace packages over the years, the setuptools hack is still the implementation of the concept that will work on the broadest range of Python versions (and I definitely know zope.interface has been using it going far back). That said, I've been meaning to try to figure out some way of supporting all three forms of namespace packages in a way that is interfering. As I said, I've had problems with this myself :/ Erik