[Distutils] namespace packages

David Cournapeau cournape at gmail.com
Sat Apr 24 07:45:29 CEST 2010

On Sat, Apr 24, 2010 at 1:32 AM, P.J. Eby <pje at telecommunity.com> wrote:

> If you don't mind trying a simple test for me, would you patch your
> pkg_resources to comment out this loop:
>            for pkg in self._get_metadata('namespace_packages.txt'):
>                if pkg in sys.modules: declare_namespace(pkg)

That looks much better. It is roughly half the time (450 ms -> 250
ms). I had a simple test set with a directory containing N empty
*.egg-info directory, and the import time was proportional to N, now
it does not matter anymore.

> This change is not backward compatible with some older packages (from years
> ago) that were not declaring their namespace packages correctly, but it has
> been announced for some time (with warnings) that such packages will not
> work with setuptools 0.7.
> (By the way, in case you're thinking this change would only affect namespace
> packages, and you don't have any, what's happening is that the
> _get_metadata() call forces a check for the *existence* of
> namespace_packages.txt in every .egg-info or .egg/EGG-INFO on your path,
> whether the file actually exists or not.  In the case of zipped eggs, this
> check is just looking in a dictionary; for actual files/directories, this is
> a stat call.)

Yes, that's exactly what I was seeing in the strace output. Is there a
design document or something else decribing how the namespace
mechanism works for setuptools ? I would like to support namespace
package in my own packaging project, but it is not clear to me what
needs to be done on my side of things.


More information about the Distutils-SIG mailing list