On Thu, Sep 19, 2013 at 12:13 AM, Benjamin Root email@example.com:
On Sep 18, 2013 11:37 PM, "PJ Eby" firstname.lastname@example.org wrote: >
On Wed, Sep 18, 2013 at 5:07 PM, Benjamin Root email@example.com wrote:
In creating a source distribution, I have found a disparity between the behaviors of distutils and setuptools with respect to package_data. As of python Issue 2279: http://bugs.python.org/issue2279, entries listed in package_data are used when building an sdist. I have verified that this works in a simple example when setup() is imported from distutils.core. However, if I import setup() from setuptools, it does not pull in the data as listed in package_data (and presumedly data_files).
You need to use the include_package_data = True flag.
I did and it didn't make a difference. Plus, shouldn't it have picked up everything that I had version controlled, anyway? Does setuptools recognize the new svn 1.7 format? Even svn 1.6 clients will refuse to do an 'svn status' on a 1.7 repo checkout. So, I think there might be multiple bugs in play.
I also don't see why I should have to use that keyword argument if distutils does fine without it. sdist isnt -- from my perspective -- a setuptools specific feature, so I had no expectation of there being such an egregious difference.
P.S. - on a related note, for a package "foo", foo.egg-info directory is created with a SOURCES.txt file. I have found that under certain situations, it seems that a successful install would result in a fully listed SOURCES.txt, and then subsequent calls to sdist seems to use that information to build seemingly correct archives. A co-working double-checking a deployment process I made did an sdist and created a source distribution without the package_data when he did a fresh checkout, but whenever I did it from my development branch, the source distribution worked fine. I haven't figured out exactly how this came about, but it seems to be tied to the SOURCES.txt file.
SOURCES.txt mostly exists so that you can safely build an sdist from an sdist, as is required by e.g. bdist_rpm, without having any revision control data on hand to guide the process. Setuptools also can insert a possibly-modified setup.cfg into an sdist for the same reason, so that if you used revision control tags to specify the version when building the sdist, any sdists rebuilt from that sdist will have the same version tags.
Yes, I understand what sources.txt is for. The issue is that it seems that it is possible for different build commands to produce different source.txt results. Since I haven't figured out how I managed to do that, it isn't the focus of my bug report.
Cheers! Ben Root
Stranger and stranger still...
So, on a hunch, I tried making sure I had the latest version of setuptools installed (mine existing install was a little over a year old). On one system, easy_install found "setuptools 1.1.6" and installed it. After that, by setup.py script behaved exactly as expected on my demo package I mocked up to test out the bug. No need for "include_package_data", or MANIFEST.in or version control, just package_data. Great!
To double-check, I tried it on another system (same version of python), and easy_install found "setuptools 1.1.6" as well and installed it. But the bug demo I made didn't work. Using "pydoc setuptools", it seems to have been pointing to distribute 0.6.28 for some reason. I retried "easy_install -U setuptools", and this time it found "distribute 0.7.3". After installing, "pydoc setuptools" now says that it is "setuptools 1.1.6". Why it didn't say that before, I haven't a clue. But the bug example package I made then worked. Don't know what to make of all that, but in the end, package_data did work as expected.
I will point out that even with setuptools 1.1.6, sdist isn't picking up all the files in version control, as I have a few other files under version control in my package that I didn't list for package_data. So, I still think there is an issue with crawling an SVN 1.7 repository.
Cheers! Ben Root