At 06:28 PM 8/8/2005 +0200, Walter Dörwald wrote:

Phillip J. Eby wrote:

...

At 07:29 PM 8/5/2005 +0200, Walter Dörwald wrote:

...

Hello Philip!

I've started to play around with your easy_install script, since managing our packages gets more complicated with each version.

After I've installed setuptools via ez_setup.py I've tried downloading the ll-xist package via "python -m easy_install ll-xist" but got a stack trace instead (see the attached stacktrace.txt). The problem seems to be the umlaut in my name. As PyPI requires UTF-8 encoded strings now, I've put a author=u"Walter Dörwald".encode("utf-8") in my setup.py when I registered the package (but a plain author=u"Walter Dörwald" in the setup.py in the package).

To work around this problem it's possible to set the system default encoding to Latin-1. I don't know if this is a problem with setuptools or distutils, but doing a simple "python setup.py install" works.

Note that this problem is a distutils problem writing PKG-INFO files; it's not anything specific to setuptools. I would recommend you use the .encode('utf-8') workaround until there is an official policy for the distutils to deal with encodings in PKG-INFO.

As PyPI espects UTF-8 now, I'd say it makes sense to use UTF-8 in package info files too.

That's probably true, but this probably needs at least some distutils-sig discussion and maybe a PEP update.

...

...

The other problem seems more severe: ll-xist is installed as the packages ll.xist, but the package init file ll/__init__.py is *not* provided by this package, but by the ll-core package instead. If I understood http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages correctly distributing subpackages via setuptools only works if the package init file is empty, which it isn't in this case.

Technically, you can do it, it's just not guaranteed that the non-empty one will be executed, unless it's first on sys.path.

...

Is there any workaround for this?

You can distribute an identical __init__.py with each distribution that shares that parent package.

I tried to avoid that, because the Windows installer then asks the user, whether files should be overwritten or not when installing the second package, and for package managers like RPM, Debian .deb or Gentoo emerge it's also problematic, if one file (even if it is empty) is provided by several packages.

Maybe the best solution, then, is to arrange things such that these installers install the packages as eggs, since this would then avoid any overwrite issues. This is pretty much how eggs are supposed to work with packaging systems in the long run.

...

and make sure that all the distributions that don't contain the _my_init module have declared a dependency to the one that does.

OK, this would work, but the clearer alternative would be to simply move everything from ll to ll.core and have ll as an empty package module again. Nevertheless this would require that each package includes the empty __init__.py again, with the problems mentioned above.

Hm. Tricky. I'll have to give this some thought, because setuptools *has* to have the __init__, since separately distributed packages are in separate directories or zipfiles, but it never occurred to me that you could hack the distutils to do partial distribution in this way. So, I never came up with a strategy for it.

Ideally, it sounds like I should have a way to ensure that when setuptools builds an egg, it should create empty __init__.py files if they don't exist. That way, you could leave out the __init__.py for other builds, but it would ensure the package exists for the egg build. The tricky part is that I can't do it in the build directory, because then it might sneak into other distributions. But it's probably doable.

...

If you want to try it, change this line in the _handle_ns routine of pkg_resources: module = sys.modules.get(packageName) or loader.load_module(packageName) to read instead: module = loader.load_module(packageName) and this will cause it to load *all* of the __init__ modules. However, the order of execution is still not guaranteed, and I'm not certain that it might not cause an __init__ to be reloaded if its grandparent directory is included twice on sys.path. Anyway, let me know if that change works for you, and I will think some more on the "load all __init__ modules" strategy. Even if I do implement it, I will want to mainly advise people to use empty __init__ modules for any new namespace packages they create.

If I've understoodd this correcty, it doesn't solve my problem, as I only have *one* __init__.py. And as setuptools puts different subpackages in different directories, that get added via .pth files, I'm not sure, if there can be a simple solution.

Well, if I create the __init__ in the zipfile, and pkg_resources executes all the __init__ files, then it should work even with your current layout, as long as there's a dependency declared from the package without the __init__ to the package with it.