[Distutils] Problems with Python eggs/setuptools

[Phillip J. Eby]

At 07:29 PM 8/5/2005 +0200, Walter Dörwald wrote:
>Hello Philip!
>
>I've started to play around with your easy_install script, since managing 
>our packages gets more complicated with each version.
>
>After I've installed setuptools via ez_setup.py I've tried downloading the 
>ll-xist package via "python -m easy_install ll-xist" but got a stack trace 
>instead (see the attached stacktrace.txt). The problem seems to be the 
>umlaut in my name. As PyPI requires UTF-8 encoded strings now, I've put a
>    author=u"Walter Dörwald".encode("utf-8")
>in my setup.py when I registered the package (but a plain
>    author=u"Walter Dörwald"
>in the setup.py in the package).
>
>To work around this problem it's possible to set the system default 
>encoding to Latin-1. I don't know if this is a problem with setuptools or 
>distutils, but doing a simple "python setup.py install" works.

Note that this problem is a distutils problem writing PKG-INFO files; it's 
not anything specific to setuptools.  I would recommend you use the 
.encode('utf-8') workaround until there is an official policy for the 
distutils to deal with encodings in PKG-INFO.


>The other problem seems more severe: ll-xist is installed as the packages 
>ll.xist, but the package init file ll/__init__.py is *not* provided by 
>this package, but by the ll-core package instead. If I understood 
>http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages 
>correctly distributing subpackages via setuptools only works if the 
>package init file is empty, which it isn't in this case.

Technically, you can do it, it's just not guaranteed that the non-empty one 
will be executed, unless it's first on sys.path.


>  Is there any workaround for this?

You can distribute an identical __init__.py with each distribution that 
shares that parent package.  If there is too much to duplicate, I would 
suggest moving the contents out to a module (e.g. _my_init.py), and then 
putting something like this in the __init__.py of each distributed package:

     import pkg_resources; del pkg_resources
     from _my_init import *

and make sure that all the distributions that don't contain the _my_init 
module have declared a dependency to the one that does.

The reason for the pkg_resources import is to ensure that it has a chance 
to set up the namespace package; if the modules were installed via a .pth 
file, and pkg_resources has not been imported yet, then the runtime system 
may not have made the package a namespace package yet, and it absolutely 
needs to be before the _my_init module gets imported, so that it will be on 
the package __path__.

On occasion I've thought of maybe executing *all* the __init__ modules, but 
it seems potentially error-prone.  If you want to try it, change this line 
in the _handle_ns routine of pkg_resources:

     module = sys.modules.get(packageName) or loader.load_module(packageName)

to read instead:

     module = loader.load_module(packageName)

and this will cause it to load *all* of the __init__ modules.  However, the 
order of execution is still not guaranteed, and I'm not certain that it 
might not cause an __init__ to be reloaded if its grandparent directory is 
included twice on sys.path.  Anyway, let me know if that change works for 
you, and I will think some more on the "load all __init__ modules" 
strategy.  Even if I do implement it, I will want to mainly advise people 
to use empty __init__ modules for any new namespace packages they create.