At 07:14 PM 10/12/2005 -0400, Robby Dermody wrote:

Hi guys,

The subject line no doubt tells you all something you already know. This morning, I didn't even know that setuputils existed, so excuse my ignorance on the topic.

Basically, I have an application that makes use of a python web application development library called nevow. The current version of nevow (0.5.0) makes use of resource_filename() to get the paths of certain static files that it uses. Things work fine when my app is not frozen, but after I freeze it with cx_freeze resource_filename() turns out to be a breakage point. I would think the same kind of thing would happen with py2exe. See the traceback at the end of this message for more info.

Once I freeze the application and run it, my sys.path is set to ['/home/robbyd/dev/inquest/dist/inquest-director-0.7.0-1/inquest-director', '/home/robbyd/dev/inquest/dist/inquest-director-0.7.0-1'] (Meaning the frozen zip file executable, 'inquest-director', and its parent path.)

Normally this works fine, due to zipimporter. However, with the static files nevow is attempting to locate with resource_filename(), it seems that the problem is that, resource_filename() wants to look into the zip file but it doesn't know how to do so. (I have not done too much research into how this function works, so I may be missing the point here.) What I'd like to know is why this is the case, and what may happen when resource_filename() gets used in more libraries whose users freeze their applications with cxfreeze/py2exe/macmillian/etc. Is this a move by setuptools to take over the world (of python packaging utilities)? ;)

Thanks in advance for any feedback that is offered. At this point I'm just unsure if this is more of a setuptools issue, or more of a cx_Freeze issue. I did take a look at setuptools and I was impressed with it, but my app has certain requirements that make cx_freeze a good choice.

You could say that it's a collision between the program's desire to use the more convenient resource_filename() function in place of the more-portable resource_stream() and resource_string() functions. The latter two functions will work just fine with arbitrary importable zip files, but resource_filename() requires either an extracted zipfile or a zipped egg.

The issue here is that resource_filename() requires the static file to be extracted to the filesystem. To be extracted to the filesystem, there needs to be a place to extract. For eggs, the extract location is partly determined by the egg's project name and version number, which are assumed to be "relatively unique", allowing the resource to be extracted to a unique location. The runtime complains in this case because it doesn't find any data that tells it what the name of the egg is that it's doing the extraction on behalf of, so it doesn't know where it should extract to.

There are two possible solutions:

1. Lobby the nevow author(s) to use resource_stream() or resource_string() instead 2. Package the .egg in the zipfile as a subdirectory

The first one is self-explanatory; it will make the package work even if it's not packaged as an egg. I suggest submitting a patch, if you can.

The second one is straightforward in principle, but perhaps not in practice, depending on the constraints of your tools. What you want to do is have a subdirectory *in your zipfile* that is named 'nevow-0.5.0-py2.4.egg', or whatever the correct name is. Inside that subdirectory, you include the original contents of the egg. Then, pkg_resources will know what egg contains the package, and thus do the extraction correctly. A zipfile that contains multiple eggs is sometimes called a "basket", as in "putting all your eggs in one basket". :)

In general, pkg_resources tries to treat zipfiles and directories inside zipfiles as equivalents to "real" directories, which means that if you have a .zip on sys.path (or even an executable with a zipfile appended to it), then pkg_resources will happily search the zipfile's root for .egg subdirectories when it's trying to resolve dependencies.

As for your world domination question, I suspect that eventually most freezing tools will just dump all the necessary eggs into a basket, since this is simpler and less likely to break than scouting out individual module dependencies. There's also an opportunity for installation-oriented tools to create "download only the bits I need" web installers, etc.