[Distutils] distutils data_files and setuptools.pkg_resources are driving me crazy

Bob Ippolito bob at redivi.com
Fri Jul 14 02:34:06 CEST 2006


On Jul 13, 2006, at 4:56 PM, Robin Bryce wrote:

>> What compelling argument?
>
> Ok, I should have said "I think there _may_ be a compelling argument.
> I'll have a go:
>
> In another thread, "best practices for creating eggs" Paul Moore said:
>
>> I work offline sufficiently often that not having local documentation
>> is frustrating. There's no standard for local docs, which is a
>> nuisance, and makes for an inconsistent story between different
>> packages, but I'd be concerned if setuptools made it more  
>> difficult to
>> bundle local docs.
>
> and Kevin Dangoor said:
>
>> Being able to get at a package's documentation after it's
>> installed would be nice.
>
> Phil said:
>
>> Note that I said above that I always put the documentation in an  
>> sdist
>> form; to obtain a package's source distribution, use:
>>
>>    easy_install -e -b somedir arg...
> <snip>
>> A standard for how to install documentation would be great
>
> I think the motivation to always package docs in sdist and arrange for
> egg and sdist down loads to appear together is really a deficiency in
> how data,eggs & pkg_resources interact. All things considered sdist is
> the easiest thing for the packager: It's a few runes in MANIFEST.in
> followed by python setup.py sdist. But you loose the ability to
> reference that data in using pkg_resources.
>
> If I put my docs, data, etc in their own egg I get some of this back.
> Something like ``package_dir = package_dir = {'foopackage': './'}``
> might be enough to fake me up a package to keep setuptools happy but
> if not I'll create one. In this instance I don't care about having an
> overly large distribution because the user has actively chosen to
> install the docs.

You said data before, not supplementary materials like docs and  
examples. Data that's consumed by the code absolutely belongs in eggs  
and shouldn't be strewn all over the place. One of the most important  
features of eggs is that they consolidate stuff. Without eggs you  
install a package and you get files thrown all over your disk without  
any way to keep track of what belongs to which package.

It would be nice if there were a convention for installing  
supplementary stuff (docs, examples, py2app/py2exe generated tools,  
xcode templates, etc.). One problem with this is that you generally  
want those things to be somewhere easily accessible by the user, and  
something based on site-packages isn't (or sys.prefix at all in some  
cases, like Mac OS X). I'm not sure how much integration with  
distutils this stuff really deserves because it's not required to  
make anything work. For now, the author could just provide a zip of  
the supplementary materials for the users to download if they want to  
look at them offline.

Another thing that setuptools is currently missing is support for  
packaging up dynamic libraries (e.g. pygame's SDL dlls) and/or  
headers (e.g. Numeric's API), but that's a bigger distutils problem.

> But all of that is less than ideal. AFAICT, if pkg_resources supported
> egg root relative resource_names, I would be able to do anything I
> want, up to an including writing egg format extensions that do more
> specialist things with data packaged in *eggs*.

Egg format extensions are already provided for by entry points and  
the EGG-INFO metadata dir. Eggs were designed to facilitate plug-ins,  
after all.

-bob



More information about the Distutils-SIG mailing list