setuptools bug report: _get_unpatched() is way too clever (and also returns the wrong module)
Bug report #1: _get_unpatched() returns a module from the system site-packages instead of the module from the PYTHONPATH which ought to have precedence, leading to setuptools-0.6c8 invoking setuptools-0.6a9 in the following stack trace: PYTHONPATH="/home/buildslave/slave-tahoe/dapper/build/support/lib/ python2.4/site-packages" python ./setup.py develop --prefix="/home/ buildslave/slave-tahoe/dapper/build/support" ... Running Nevow-0.9.18/setup.py -q bdist_egg --dist-dir /tmp/ easy_install-rk83KQ/Nevow-0.9.18/egg-dist-tmp-nJ8GSy Traceback (most recent call last): ... File "/home/buildslave/slave-tahoe/dapper/build/ setuptools-0.6c8.egg/setuptools/command/develop.py", line 102, in install_for_development File "/usr/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/ easy_install.py", line 518, in process_distribution Full details: https://dev.allmydata.com/buildbot-tahoe/builders/dapper/builds/1427/ steps/compile/logs/stdio Bug report #2: _get_unpatched() is way too clever for me to spend time trying to understand and debug right now: def _get_unpatched(cls): """Protect against re-patching the distutils if reloaded Also ensures that no other distutils extension monkeypatched the distutils first. """ What the heck? I don't want to think about that, so instead we're delisting Ubuntu Dapper as a supported platform, since it is the only one of our supported platforms on which we have this problem. Generalized complaint: There are a lot of things that are too clever in setuptools/ easy_install/eggs. Cleverness makes it work in more situations, which is good, but it also means that it is harder to know when it will fail or to understand why it failed. A simpler system that satisfied 90% of the use cases while failing in more obvious ways would be appreciated. (Provided, of course, that those 90% of use cases include the ones that I really care about.) See my earlier post which proposes such a simplification: http://mail.python.org/pipermail/python-dev/2008-March/078243.html Bug report #3: setuptools/easy_install/eggs needs a bug tracker. (Also, of course, a buildbot and a rich suite of automated unit tests.) Some of this message may have been inappropriate for distutils-sig, but what else am I going to do with this issue before I delist Dapper and forget all about it? I would be willing to host a trac instance to serve as a wiki and issue tracker for setuptools, and a buildbot. (But I don't want to have anything to do with subversion -- I much prefer darcs.) Regards, Zooko
At 06:09 PM 3/26/2008 -0700, zooko wrote:
Bug report #1: ... PYTHONPATH="/home/buildslave/slave-tahoe/dapper/build/support/lib/ python2.4/site-packages" python ./setup.py develop --prefix="/home/ buildslave/slave-tahoe/dapper/build/support" ... Running Nevow-0.9.18/setup.py -q bdist_egg --dist-dir /tmp/ easy_install-rk83KQ/Nevow-0.9.18/egg-dist-tmp-nJ8GSy Traceback (most recent call last): ... File "/home/buildslave/slave-tahoe/dapper/build/ setuptools-0.6c8.egg/setuptools/command/develop.py", line 102, in install_for_development File "/usr/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/ easy_install.py", line 518, in process_distribution
Full details:
https://dev.allmydata.com/buildbot-tahoe/builders/dapper/builds/1427/ steps/compile/logs/stdio
That URL requires login credentials that I don't have. And I need the rest of the traceback to make any sense of this.
Bug report #2:
_get_unpatched() is way too clever for me to spend time trying to understand and debug right now: ... What the heck? I don't want to think about that, so instead we're delisting Ubuntu Dapper as a supported platform, since it is the only one of our supported platforms on which we have this problem.
Looking at what limited information you've given me, I doubt the problem has anything to do with this function, whose operation is ridiculously simple - it finds the first base class that wasn't defined by setuptools, and *was* defined by the distutils. Since the distutils don't define an easy_install *or* develop command, neither one uses _get_unpatched and there should thus be no way that this is causing the weird import situation you have. This is 99.9% unlikely to be related in *any* way to _get_unpatched(), and I'm only hedging the .1% because I'm cautious by nature. :) (Well, that, and there's not a lot of info to go on here.)
On Mar 26, 2008, at 6:56 PM, Phillip J. Eby wrote:
https://dev.allmydata.com/buildbot-tahoe/builders/dapper/builds/ 1427/steps/compile/logs/stdio
That URL requires login credentials that I don't have. And I need the rest of the traceback to make any sense of this.
My apologies -- I meant: http://allmydata.org/buildbot/builders/dapper/builds/1427/steps/ compile/logs/stdio
Looking at what limited information you've given me, I doubt the problem has anything to do with this function, whose operation is ridiculously simple - it finds the first base class that wasn't defined by setuptools, and *was* defined by the distutils. Since the distutils don't define an easy_install *or* develop command, neither one uses _get_unpatched and there should thus be no way that this is causing the weird import situation you have.
This is 99.9% unlikely to be related in *any* way to _get_unpatched (), and I'm only hedging the .1% because I'm cautious by nature. :) (Well, that, and there's not a lot of info to go on here.)
Okay, then I admit that _get_unpatched()'s cleverness may not be the cause of this bug, but it was certainly the cause of Brian Warner and me ceasing to debug this, since we needed to grok _get_unpatched() in order to determine that it was not the cause of the problem. My offer to host an issue tracker for setuptools/easy_install/ pkg_resources/eggs is still open. It could be a good way to organize and remember knowledge. Regards, Zooko
At 07:25 PM 3/26/2008 -0700, zooko wrote:
On Mar 26, 2008, at 6:56 PM, Phillip J. Eby wrote:
https://dev.allmydata.com/buildbot-tahoe/builders/dapper/builds/ 1427/steps/compile/logs/stdio
That URL requires login credentials that I don't have. And I need the rest of the traceback to make any sense of this.
My apologies -- I meant:
http://allmydata.org/buildbot/builders/dapper/builds/1427/steps/ compile/logs/stdio
Okay, looking at that, my guess is that the problem is due to the fact that setuptools 0.6c8 and its 'site.py' aren't installed until the "develop" command is run, and somehow causing the setuptools package to get split across two locations. I would suggest that your makefile should ensure the desired version of setuptools is installed before anything else. If that doesn't fix it, my next guess would be that Nevow's setup script is altering sys.path in some way, or that that or some other setup script tries to forcibly evict or reload setuptools. That's about the only other way I can see for this to get split across two locations like that. Are you using an ez_setup anywhere in there that monkeys with sys.modules?
Okay, then I admit that _get_unpatched()'s cleverness may not be the cause of this bug, but it was certainly the cause of Brian Warner and me ceasing to debug this, since we needed to grok _get_unpatched() in order to determine that it was not the cause of the problem.
I don't really see how it relates in any way, since it's not listed in the traceback. In any event, the traceback itself also appears unrelated to whatever problem Nevow's setup is having.
zooko wrote:
I would be willing to host a trac instance to serve as a wiki and issue tracker for setuptools, and a buildbot. (But I don't want to have anything to do with subversion -- I much prefer darcs.)
I've previously volunteered Enthought to do about the same thing (except to include svn with the Trac instance) but there wasn't much response beyond Jeff Rush saying he was already setting one up, and the response by Phillip to use the python dev bug-tracker and svn instead. (No one previously talked about buildbot but that makes great sense.) I haven't seen any follow-up on those discussions though. Did I miss it? ;-) I'd really like to get this resolved, along with the issues of who can commit to the codebase, so that more people can start working on some of this stuff being discussed. Phillip, I think everyone is waiting for you to provide direction so can you make some decisions about how setuptools evolves to a larger community effort? -- Dave
At 11:40 PM 4/1/2008 -0500, Dave Peterson wrote:
I've previously volunteered Enthought to do about the same thing (except to include svn with the Trac instance) but there wasn't much response beyond Jeff Rush saying he was already setting one up, and the response by Phillip to use the python dev bug-tracker and svn instead. ...
Phillip, I think everyone is waiting for you to provide direction so can you make some decisions about how setuptools evolves to a larger community effort?
As I believe you just pointed out, I already did that. :)
Phillip J. Eby wrote:
At 11:40 PM 4/1/2008 -0500, Dave Peterson wrote:
I've previously volunteered Enthought to do about the same thing (except to include svn with the Trac instance) but there wasn't much response beyond Jeff Rush saying he was already setting one up, and the response by Phillip to use the python dev bug-tracker and svn instead. ...
Phillip, I think everyone is waiting for you to provide direction so can you make some decisions about how setuptools evolves to a larger community effort?
As I believe you just pointed out, I already did that. :)
Egggcellent. Doing a search for 'setuptools' in the issue tracker only returns two open issues currently. I'm guessing I'm not catching all the issues as a result. It would probably help if we had a clear component to use (i.e. Setuptools). Anyone have any idea who to contact to get that done? I've so far had no luck reading the info on www.python.org/dev looking for this sort of thing. Phillip, I don't believe you responded to either my, nor Jeff Rush's, volunteering for being patch reviewers. At least I haven't seen any response -- perhaps that's you trying to tell me something? :-) What's the process you want to follow on this? Ideally things should be opened up where more people are 'blessed' to commit directly, but I'm not sure how you want to do the 'bless'ing part. Finally, what is the build and test process you've been following? Is there a buildbot that's building and publishing results somewhere or does that also need to be set up? -- Dave
Dave Peterson wrote:
Phillip J. Eby wrote:
At 11:40 PM 4/1/2008 -0500, Dave Peterson wrote:
I've previously volunteered Enthought to do about the same thing (except to include svn with the Trac instance) but there wasn't much response beyond Jeff Rush saying he was already setting one up, and the response by Phillip to use the python dev bug-tracker and svn instead. ...
Phillip, I think everyone is waiting for you to provide direction so can you make some decisions about how setuptools evolves to a larger community effort? As I believe you just pointed out, I already did that. :)
Egggcellent.
Doing a search for 'setuptools' in the issue tracker only returns two open issues currently. I'm guessing I'm not catching all the issues as a result. It would probably help if we had a clear component to use (i.e. Setuptools). Anyone have any idea who to contact to get that done? I've so far had no luck reading the info on www.python.org/dev looking for this sort of thing.
Hi Dave, At PyCon I asked the core team about using the Python tracker for setuptools but Brett Cannon said that no software outside of the Python core is currently using it and it wouldn't be appropriate for setuptools to do so. They were willing to give us our own instance of a tracker, similar to how they have for Jython. Martin v. Löwis has been in the process since PyCon of doing so, but is busy and apparently the Roundup software used is tricky to configure for this and,f for the email address, requires ISP intervention. The plan when this is done is to have a tracker at http://bugs.python.org/setuptools/ with an email address for submission/commenting of setuptools@bugs.python.org. Sorry for the delay - I didn't realize this would be such a big deal to get started. We could also start using Launchpad instantly, but I figured reaching concensus to do so, with Phillip and others more familiar with the Python tracker, would be difficult. -Jeff
Jeff Rush wrote:
Dave Peterson wrote:
Phillip J. Eby wrote:
At 11:40 PM 4/1/2008 -0500, Dave Peterson wrote:
I've previously volunteered Enthought to do about the same thing (except to include svn with the Trac instance) but there wasn't much response beyond Jeff Rush saying he was already setting one up, and the response by Phillip to use the python dev bug-tracker and svn instead.
...
Phillip, I think everyone is waiting for you to provide direction so can you make some decisions about how setuptools evolves to a larger community effort?
As I believe you just pointed out, I already did that. :)
Egggcellent.
Doing a search for 'setuptools' in the issue tracker only returns two open issues currently. I'm guessing I'm not catching all the issues as a result. It would probably help if we had a clear component to use (i.e. Setuptools). Anyone have any idea who to contact to get that done? I've so far had no luck reading the info on www.python.org/dev looking for this sort of thing.
Hi Dave,
At PyCon I asked the core team about using the Python tracker for setuptools but Brett Cannon said that no software outside of the Python core is currently using it and it wouldn't be appropriate for setuptools to do so. They were willing to give us our own instance of a tracker, similar to how they have for Jython.
Martin v. Löwis has been in the process since PyCon of doing so, but is busy and apparently the Roundup software used is tricky to configure for this and,f for the email address, requires ISP intervention. The plan when this is done is to have a tracker at http://bugs.python.org/setuptools/ with an email address for submission/commenting of setuptools@bugs.python.org.
Sorry for the delay - I didn't realize this would be such a big deal to get started. We could also start using Launchpad instantly, but I figured reaching concensus to do so, with Phillip and others more familiar with the Python tracker, would be difficult.
While I'm glad to hear there has been progress, I'm concerned we risk losing momentum by waiting. Can we at least start by using their wiki to capture some thoughts on priorities and roadmap? Or does that need to be separate too? Or does Phillip already have that somewhere I haven't seen? -- Dave
Martin v. Löwis has been in the process since PyCon of doing so, but is busy and apparently the Roundup software used is tricky to configure for this and,f for the email address, requires ISP intervention. The plan when this is done is to have a tracker at http://bugs.python.org/setuptools/ with an email address for submission/commenting of setuptools@bugs.python.org. Sorry for the delay - I didn't realize this would be such a big deal to get started. We could also start using Launchpad instantly, but I figured reaching concensus to do so, with Phillip and others more familiar with the Python tracker, would be difficult.
While I'm glad to hear there has been progress, I'm concerned we risk losing momentum by waiting.
Can we at least start by using their wiki to capture some thoughts on priorities and roadmap? Or does that need to be separate too? Or does Phillip already have that somewhere I haven't seen?
Just to give us all a kick in the pants, I set up a trac instance and imported setuptools revision control history into it: http://allmydata.org/trac/setuptools It took me only a few minutes -- I'm getting familiar with the process of setting up trac instances for open source projects. Hm, the timeline for the last 90 days: http://allmydata.org/trac/setuptools/timeline? from=2008-04-04&daysback=90&ticket=on&changeset=on&milestone=on&update=U pdate The issue tracker is currently empty: http://allmydata.org/trac/setuptools/report/1 You can register and start scribbling on the wiki if you like -- anything you write there can be copied over to a new site once it is operational. The "buildbot" link yields a 404 Not Found -- I haven't set up a buildbot yet. Whether or not we use trac, and whether or not we use hosting provided by Allmydata, Inc., on http://allmydata.org , we should definitely set up a buildbot. Regards, Zooko
participants (4)
-
Dave Peterson -
Jeff Rush -
Phillip J. Eby -
zooko