[Distutils] Distutils 2.7/3.1 and setuptools stack overflow

["Martin v. Löwis"]

>> I have investigated the problem of mutual recursion between setuptools
>> and distutils a little. I read a message somewhere that you didn't
>> understand the problem, so here is my analysis:
>>
>> 1. build_py.data_files gets accessed, and is delayed-computed in
>>    setuptools
>> 2. build_py tries to load the manifest, and goes to egg_info for that
>> 3. egg_info invokes add_defaults for the manifest
>> 4. manifest_maker.add_defaults invokes sdist.add_defaults
>> 5. in response to issue 2279, add_defaults now also adds build_py
>>    data_files and package_data to the default files.
>>
>> As a work-around, I have now copied the 2.6 code of sdist.add_defaults
>> into manifest_maker.sdist_add_defaults.
> 
> I don't understand.  Where did you do this?  In setuptools or distutils?

In setuptools, more precisely, in the manifest_maker class.

>> In the long run, I think setuptools should revise its computation of
>> data_files. Couldn't you simply use distutils' get_data_files()?
> 
> Setuptools supports an 'include_package_data' option that automatically
> includes any revision-controlled files found in package directories. 
> Distutils doesn't.  The somewhat-recursive aspect is that when an sdist
> is used instead of a checkout, the previous revision control info is not
> available, so setuptools must use the previously-computed list of
> included files in order to find out which files are "package data".

So how does that interwork with the list of data files specified in
an explicit package_data setting of setup.py?

Also, if all it wants is to compute the list of version-controlled
files, and deduce data_files from that, why does it have to go through
the manifest computation?

> It's possible that this could be resolved in some other way, but IIUC
> Tarek has already implemented a setuptools patch to fix this, which has
> been applied to the "Distribute" fork of setuptools.

I found that patch afterwards. It is similar to mine, except that he
modifies setuptools' sdist class, instead of manifest_maker. So my
change is less intrusive, but his change possibly fixes other cases
as well.

Regards,
Martin