Re: [Distutils] distutils data_files and setuptools.pkg_resources are driving me crazy
At 08:52 PM 7/13/2006 +0100, Robin Bryce wrote:
file:setup.cfg [egg_info] egg_base=./ # because I guessed (incorrectly) that this would help.
This option controls where the .egg-info *build* directory will be, and *nothing* else. But I think you figured that out. :) Meanwhile, I think the solution to your overall problem will require either expanding the metadata API to allow file access, or else adding some kind of path facility to the resource API (e.g. '/'-prefixed paths as you suggested) to access a 'data/' subdirectory of the egg-info directory. I'll look into this option in setuptools 0.7, although I think it will also require some work on the install_data command. There's also a potentially significant limitation in that you won't be able to look up non-package resources using package names. That is, you're not going to be able to do resource_filename('foo.bar', '/baz'). Instead, you would have to do resource_filename(Requirement.parse('MyProject'), '/baz'). Again, this is an unfortunate quirk caused by the need to support legacy package layouts for system packaging tools such as RPM, Debian, etc.
This option controls where the .egg-info *build* directory will be and *nothing* else
Yes. :-) Did you notice, in my unfortunately long initial post, the issue I had with script generation when the build directory was in a non default location ? egg_base=./ seemed to prevent the installation of console_scripts. I'm planing on spending today attempting a test case patch to show this. You'd save me some time if you could say "this is not a bug" :-) [apologies, quoting you in reverse order here]
I'll look into this option in setuptools 0.7, although I think it will also require some work on the install_data command. There's also a potentially significant limitation in that you won't be able to look up non-package resources using package names. That is, you're not going to be able to do resource_filename('foo.bar', '/baz'). Instead, you would have to do resource_filename(Requirement.parse('MyProject'), '/baz').
I don't think this is a limitation. As the package author I have gone to some trouble to say "this data is not package data". I would not then expect (or want) to be able to reference that data using the ``package`` bit of ``resouce_string(package_or_requirement, resource_name)``. If resource_name referes to non package data then I *expect* Requirement.parse('MyProject') to be its name space when referenced by my program.
Meanwhile, I think the solution to your overall problem will require either expanding the metadata API to allow file access, or else adding some kind of path facility to the resource API (e.g. '/'-prefixed paths as you suggested) to access a 'data/' subdirectory of the egg-info directory.
Yes, I agree. I'll have a stab at thinking this through: (Aside: I think it's a *good* thing that eggs don't attempt to be a 'full blown' virtualised file system - that would involve too much magic for my tastes, refactored or not ;-). There are at least two 'file' systems in play here: the package authors development source tree and the eggs collection of resource_names. All information to produce the latter is available, or could be available, to setuptools once ``python setup.py egg_info`` has run. A key advantage (for me at least) of ``python setup.py develop`` is that it produces the .egg-info directory but does *not* trigger a full build of my sources. Specifically, it is most advantageous where my source tree has a lot of data that would otherwise be copied into the build tree. Being able to avoid this copy and still run something that is pretty close to what my end users will run is *very* cool. It would be a shame to loose this for non packaged data files. Which suggests that .egg-info would need to remember enough information about the source tree in order to allow pkg_resources to perform a reverse mapping for resource_filename(Requirement.parse('MyProject'), '/baz') I can not see a reasonable way to support: python lib/MyProject/entrypoint.py --list-non-packaged-data. Ie, expecting pkg_resources to perform this reverse mapping *before* .egg-info is built. If python setup.py develop does *not* copy data_files into place, I don't think it's unreasonable for pkg_resources to assume EGG-INFO is always available. For all 'installed' cases I think setuptools can get at EGG-INFO without guess work. For python setup.py develop, I think heuristics in pkg_resources, that look for .egg-info relative to MyPackage are acceptable. boost bjam does something allong these lines when it looks for boost-build.jam. The confusion I had initially with egg_base was that I guessed (incorrectly) that it could be used as a way to preserve the 'prefix' information and make it available in the resultant .egg-info. I find my self *really* wanting to help get this into 0.7. I have both the time and the need to work on this. I'm happy to have a stab blind and contribute the results. Alternately, if you spec it, I'd have no trouble letting you direct my typing fingers :-) Cheers, Robin
participants (2)
-
Phillip J. Eby
-
Robin Bryce