[Distutils] Deviation between distutils and setuptools over package_data

[Benjamin Root]

On Thu, Sep 19, 2013 at 12:13 AM, Benjamin Root <ben.v.root at gmail.com>wrote:

>
> On Sep 18, 2013 11:37 PM, "PJ Eby" <pje at telecommunity.com> wrote:
> >
> > On Wed, Sep 18, 2013 at 5:07 PM, Benjamin Root <ben.v.root at gmail.com>
> wrote:
> > > In creating a source distribution, I have found a disparity between the
> > > behaviors of distutils and setuptools with respect to package_data. As
> of
> > > python Issue 2279: http://bugs.python.org/issue2279, entries listed in
> > > package_data are used when building an sdist. I have verified that this
> > > works in a simple example when setup() is imported from distutils.core.
> > > However, if I import setup() from setuptools, it does not pull in the
> data
> > > as listed in package_data (and presumedly data_files).
> >
> > You need to use the include_package_data = True flag.
> >
>
> I did and it didn't make a difference. Plus, shouldn't it have picked up
> everything that I had version controlled, anyway? Does setuptools recognize
> the new svn 1.7 format? Even svn 1.6 clients will refuse to do an 'svn
> status' on a 1.7 repo checkout. So, I think there might be multiple bugs in
> play.
>
> I also don't see why I should have to use that keyword argument if
> distutils does fine without it. sdist isnt -- from my perspective -- a
> setuptools specific feature, so I had no expectation of there being such an
> egregious difference.
>
> >
> > > P.S. - on a related note, for a package "foo", foo.egg-info directory
> is
> > > created with a SOURCES.txt file. I have found that under certain
> situations,
> > > it seems that a successful install would result in a fully listed
> > > SOURCES.txt, and then subsequent calls to sdist seems to use that
> > > information to build seemingly correct archives. A co-working
> > > double-checking a deployment process I made did an sdist and created a
> > > source distribution without the package_data when he did a fresh
> checkout,
> > > but whenever I did it from my development branch, the source
> distribution
> > > worked fine. I haven't figured out exactly how this came about, but it
> seems
> > > to be tied to the SOURCES.txt file.
> >
> > SOURCES.txt mostly exists so that you can safely build an sdist from
> > an sdist, as is required by e.g. bdist_rpm, without having any
> > revision control data on hand to guide the process.  Setuptools also
> > can insert a possibly-modified setup.cfg into an sdist for the same
> > reason, so that if you used revision control tags to specify the
> > version when building the sdist, any sdists rebuilt from that sdist
> > will have the same version tags.
>
> Yes, I understand what sources.txt is for. The issue is that it seems that
> it is possible for different build commands to produce different source.txt
> results. Since I haven't figured out how I managed to do that, it isn't the
> focus of my bug report.
>
> Cheers!
> Ben Root
>
Stranger and stranger still...

So, on a hunch, I tried making sure I had the latest version of setuptools
installed (mine existing install was a little over a year old). On one
system, easy_install found "setuptools 1.1.6" and installed it. After that,
by setup.py script behaved exactly as expected on my demo package I mocked
up to test out the bug. No need for "include_package_data", or MANIFEST.in
or version control, just package_data. Great!

To double-check, I tried it on another system (same version of python), and
easy_install found "setuptools 1.1.6" as well and installed it. But the bug
demo I made didn't work. Using "pydoc setuptools", it seems to have been
pointing to distribute 0.6.28 for some reason. I retried "easy_install -U
setuptools", and this time it found "distribute 0.7.3". After installing,
"pydoc setuptools" now says that it is "setuptools 1.1.6". Why it didn't
say that before, I haven't a clue. But the bug example package I made then
worked. Don't know what to make of all that, but in the end, package_data
did work as expected.

I will point out that even with setuptools 1.1.6, sdist isn't picking up
all the files in version control, as I have a few other files under version
control in my package that I didn't list for package_data. So, I still
think there is an issue with crawling an SVN 1.7 repository.

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130919/8ebe9abd/attachment.html>