Mutual recursion between setuptools and distutils with Python 2.7
Hello, I'm not sure where to post this (distutils or setuptools bug?), so I'm trying here first. Using the CPython trunk, there's an infinite recursion due to a mutual recursion between setuptools and distutils. Here is a snippet of a traceback (trimmed for obvious reasons): $ usr/bin/easy_install -Z -i http://www.turbogears.org/2.0/downloads/current/index tg.devtools Searching for tg.devtools Reading http://www.turbogears.org/2.0/downloads/current/index/tg.devtools/ Best match: tg.devtools 2.0 Downloading http://www.turbogears.org/2.0/downloads/current/tg.devtools-2.0.tar.gz Processing tg.devtools-2.0.tar.gz Running tg.devtools-2.0/setup.py -q bdist_egg --dist-dir /home/antoine/tmp/easy_install-AkgVEJ/tg.devtools-2.0/egg-dist-tmp-DQjVL4 Traceback (most recent call last): File "usr/bin/easy_install", line 8, in <module> load_entry_point('setuptools==0.6c9', 'console_scripts', 'easy_install')() File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1671, in main File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1659, in with_ei_usage File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1675, in <lambda> File "/home/antoine/cpython/__svn__/Lib/distutils/core.py", line 149, in setup dist.run_commands() File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 929, in run_commands self.run_command(cmd) File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in run_command cmd_obj.run() File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 211, in run File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 446, in easy_install File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 476, in install_item File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 655, in install_eggs File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 930, in build_and_install File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 919, in run_setup File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 27, in run_setup File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 63, in run File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 29, in <lambda> File "setup.py", line 53, in <module> return None File "/home/antoine/cpython/__svn__/Lib/distutils/core.py", line 149, in setup dist.run_commands() File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 929, in run_commands self.run_command(cmd) File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in run_command cmd_obj.run() File "build/bdist.linux-x86_64/egg/setuptools/command/bdist_egg.py", line 167, in run File "/home/antoine/cpython/__svn__/Lib/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in run_command cmd_obj.run() File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 177, in run File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 252, in find_sources File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 306, in run File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 330, in add_defaults File "/home/antoine/cpython/__svn__/Lib/distutils/command/sdist.py", line 288, in add_defaults for pkg, src_dir, build_dir, filenames in build_py.data_files: File "build/bdist.linux-x86_64/egg/setuptools/command/build_py.py", line 39, in __getattr__ File "build/bdist.linux-x86_64/egg/setuptools/command/build_py.py", line 44, in _get_data_files File "build/bdist.linux-x86_64/egg/setuptools/command/build_py.py", line 92, in analyze_manifest File "/home/antoine/cpython/__svn__/Lib/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in run_command cmd_obj.run() [...] File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 177, in run File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 252, in find_sources File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 305, in run File "/home/antoine/cpython/__svn__/Lib/distutils/filelist.py", line 45, in findall self.allfiles = findall(dir) File "build/bdist.linux-x86_64/egg/setuptools/__init__.py", line 71, in findall File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk for x in walk(path, topdown, onerror, followlinks): File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk for x in walk(path, topdown, onerror, followlinks): File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk for x in walk(path, topdown, onerror, followlinks): File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk for x in walk(path, topdown, onerror, followlinks): File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk for x in walk(path, topdown, onerror, followlinks): File "/home/antoine/cpython/__svn__/Lib/os.py", line 284, in walk if isdir(join(top, name)): File "/home/antoine/cpython/__svn__/Lib/genericpath.py", line 41, in isdir st = os.stat(s) File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 87, in wrap RuntimeError: maximum recursion depth exceeded
Hello Antoine, Yes, I have already mentioned the problem to Phillip a few months ago (in February). Setuptools no longer works with Distutils trunk. it happens because setuptools build_py implementation has a __getattr__ that computes data_file: def __getattr__(self,attr): if attr=='data_files': # lazily compute data files self.data_files = files = self._get_data_files(); return files return _build_py.__getattr__(self,attr) But Distutils sdist command loops over build_py.data_files to add them in the MANIFEST file, because this attribute should be computed by finalize_command. And this is done when sdist.add_default is called, so it loops recursively and dies see "for pkg, src_dir, build_dir, filenames in build_py.data_files: in" http://svn.python.org/view/python/trunk/Lib/distutils/command/sdist.py?r1=68951&r2=69692 I think Setuptools needs to change its strategy when it builds the file lists to include. Probably by splitting the work in two distinct phases and remove the getattr hack. Now the major problem with this is : the last setuptools commit is 7 months old, and right now people are trying to obtain a new setuptools version for svn 1.6 compatiblity for quite a time now. So I fear that setuptools might end up broken when 2.7 and 3.1 is out. I can try to help on that, but I need two things from Phillip: 1/ what he wants to do on setuptools side to fix that problem 2/ blessing me to do it, and do a release with that change + the patches that are waiting for some time, that he is OK to add. ++ Tarek -- Tarek Ziadé | http://ziade.org
Tarek Ziadé <ziade.tarek <at> gmail.com> writes:
Now the major problem with this is : the last setuptools commit is 7 months old, and right now people are trying to obtain a new setuptools version for svn 1.6 compatiblity for quite a time now.
I don't have any take in this, but perhaps a solution would be a friendly fork of setuptools. Apparently, it's not like there would be much competition between both branches. :) Not being able to test setuptools-requiring software with 2.7 is rather annoying. Regards Antoine.
On Mon, Jun 1, 2009 at 12:41 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Tarek Ziadé <ziade.tarek <at> gmail.com> writes:
Now the major problem with this is : the last setuptools commit is 7 months old, and right now people are trying to obtain a new setuptools version for svn 1.6 compatiblity for quite a time now.
I don't have any take in this, but perhaps a solution would be a friendly fork of setuptools. Apparently, it's not like there would be much competition between both branches. :)
If I fork setuptools to fix those problem, I will have to release it under another name, and ask people to change their setup.py file. This can lead to a messy situation in the Python community imho. People are already quite disturbed by the fact that we have distutils *and* setuptools I think the solution is that Phillip bless someone that have time to spend on it, to work on fixing those problems in setuptools. I have volunteered to maintain distutils, and I am currently pushing the good bits of setuptools in Distutils. I can work a bit on setuptools side too since it's located on the same Subversion. But I need Phillip insights and blessing for that.
Not being able to test setuptools-requiring software with 2.7 is rather annoying.
Yes, I agree. Regards Tarek -- Tarek Ziadé | http://ziade.org
Tarek Ziadé <ziade.tarek <at> gmail.com> writes:
If I fork setuptools to fix those problem, I will have to release it under another name, and ask people to change their setup.py file.
Obviously I'm playing devil's advocate a bit, but a distribution doesn't have to be named "setuptools" to provide a Python package named "setuptools" ;)
I think the solution is that Phillip bless someone that have time to spend on it, to work on fixing those problems in setuptools.
It seems to be the issue here: PJE accepting to delegate maintenance to another person or group of people. This would be a reasonable attitude. Otherwise someone *will* have to fork it eventually... There's no third alternative, too many people are relying on it right now. Regards Antoine.
On Mon, Jun 1, 2009 at 4:37 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Tarek Ziadé <ziade.tarek <at> gmail.com> writes:
If I fork setuptools to fix those problem, I will have to release it under another name, and ask people to change their setup.py file.
Obviously I'm playing devil's advocate a bit, but a distribution doesn't have to be named "setuptools" to provide a Python package named "setuptools" ;)
Yes, so this will somehow overwrite an existing installation of setuptools
I think the solution is that Phillip bless someone that have time to spend on it, to work on fixing those problems in setuptools.
It seems to be the issue here: PJE accepting to delegate maintenance to another person or group of people. This would be a reasonable attitude.
+1
Otherwise someone *will* have to fork it eventually... There's no third alternative, too many people are relying on it right now.
Regards
Antoine.
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Tarek Ziadé | http://ziade.org
participants (2)
-
Antoine Pitrou -
Tarek Ziadé