Re: [Distutils] Distutils 2.7/3.1 and setuptools stack overflow
At 03:32 PM 7/19/2009 +0200, Martin v. Löwis wrote:
Hi Phillip,
I have investigated the problem of mutual recursion between setuptools and distutils a little. I read a message somewhere that you didn't understand the problem, so here is my analysis:
1. build_py.data_files gets accessed, and is delayed-computed in setuptools 2. build_py tries to load the manifest, and goes to egg_info for that 3. egg_info invokes add_defaults for the manifest 4. manifest_maker.add_defaults invokes sdist.add_defaults 5. in response to issue 2279, add_defaults now also adds build_py data_files and package_data to the default files.
As a work-around, I have now copied the 2.6 code of sdist.add_defaults into manifest_maker.sdist_add_defaults.
I don't understand. Where did you do this? In setuptools or distutils?
In the long run, I think setuptools should revise its computation of data_files. Couldn't you simply use distutils' get_data_files()?
Setuptools supports an 'include_package_data' option that automatically includes any revision-controlled files found in package directories. Distutils doesn't. The somewhat-recursive aspect is that when an sdist is used instead of a checkout, the previous revision control info is not available, so setuptools must use the previously-computed list of included files in order to find out which files are "package data". It's possible that this could be resolved in some other way, but IIUC Tarek has already implemented a setuptools patch to fix this, which has been applied to the "Distribute" fork of setuptools. I am seriously considering simply blessing that fork *as* setuptools, at least with respect to the 0.6 maintenance branch, and turning over its stewardship to the Package Inquisition, or whatever they're calling themselves. ;-) ("The Society For Packaging Things On Top Of Other Things" would probably be another good Monty Python-inspired name for the group.)
On Sun, Jul 19, 2009 at 9:14 PM, P.J. Eby<pje@telecommunity.com> wrote:
It's possible that this could be resolved in some other way, but IIUC Tarek has already implemented a setuptools patch to fix this, which has been applied to the "Distribute" fork of setuptools.
Yes I came up with the same analysis, and I have applied the same fix Martin is mentioning in Distribute, which is also the latest patch added in setuptools tracker some times ago.
I have investigated the problem of mutual recursion between setuptools and distutils a little. I read a message somewhere that you didn't understand the problem, so here is my analysis:
1. build_py.data_files gets accessed, and is delayed-computed in setuptools 2. build_py tries to load the manifest, and goes to egg_info for that 3. egg_info invokes add_defaults for the manifest 4. manifest_maker.add_defaults invokes sdist.add_defaults 5. in response to issue 2279, add_defaults now also adds build_py data_files and package_data to the default files.
As a work-around, I have now copied the 2.6 code of sdist.add_defaults into manifest_maker.sdist_add_defaults.
I don't understand. Where did you do this? In setuptools or distutils?
In setuptools, more precisely, in the manifest_maker class.
In the long run, I think setuptools should revise its computation of data_files. Couldn't you simply use distutils' get_data_files()?
Setuptools supports an 'include_package_data' option that automatically includes any revision-controlled files found in package directories. Distutils doesn't. The somewhat-recursive aspect is that when an sdist is used instead of a checkout, the previous revision control info is not available, so setuptools must use the previously-computed list of included files in order to find out which files are "package data".
So how does that interwork with the list of data files specified in an explicit package_data setting of setup.py? Also, if all it wants is to compute the list of version-controlled files, and deduce data_files from that, why does it have to go through the manifest computation?
It's possible that this could be resolved in some other way, but IIUC Tarek has already implemented a setuptools patch to fix this, which has been applied to the "Distribute" fork of setuptools.
I found that patch afterwards. It is similar to mine, except that he modifies setuptools' sdist class, instead of manifest_maker. So my change is less intrusive, but his change possibly fixes other cases as well. Regards, Martin
participants (3)
-
"Martin v. Löwis" -
P.J. Eby -
Tarek Ziadé