[Distutils] namespace packages

[David Cournapeau]

On Fri, Apr 23, 2010 at 5:54 PM, Tarek Ziadé <ziade.tarek at gmail.com> wrote:

> I am not sure what you are defining as "complicated". While pkg_resources
> is hard to read and it's a project on its own with many other
> features, the use case
> we are talking about here is dead simple:
>
>  scan all sys.path entries to look for .egg and .egg-info files/directories.

My knowledge may be lacking here, but doen't pkg_resources need to
scan things beyond egg-info (to get namespace_package.txt, presumably)
?

Scanning egg/egg-info is easy, but that does not explain most
additional syscalls caused by pkg_resources import.

> We have an implementation
> (http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py) that is being
> moved
> to distutils2, and that will probably land in the stdlib if PEP 376 is accepted.

pkgutil.py implements everything needed for setuptools namespace
package ? From reading the discussions around PEP 382, it seemed much
more complicated.

> Hey, I barely did any C since college, I am going to give it a shot
> and see if it goes faster :)

If pkgutils.py is indeed all that is needed to support setuptools
namespace package, I am willing to look at the code to see if it can
be sped up. I really doubt that C should be used - if it takes so much
time for a couple of dozens packages, it is much more likely a design
problem than an implementation problem.

cheers,

David