Brand new, clean server, set up from scratch, buildbot still fails
http://pastebin.com/m69796bc5 This is running on a bare system, after figuring out which development tools were needed, all the version control systems necessary etc. I have all that captured in my Fabric script so we're all set for being able to duplicate it. I have NOT installed anything into the system Python at all. In fact, other than the tools necessary to pull the products out of VCS, and the compilers and such, I haven't installed (or done) anything to the system at all. That's kind of the point. We want to be able to go from bare metal to running buildbot completely automatically. I don't quite understand what this thing is supposed to be doing but I'm sure it's not doing everything it needs to do to get itself "bootstrapped"; on a fresh Ubuntu install, anyway. BTW, on my OS X development system, running the system Python 2.6.1, it doesn't have any trouble at all except for this little gem early in the process. /usr/bin/python "-c" "from setuptools.command.easy_install import main; main()" "-mUNxd" "/Users/ssteiner/Dropbox/work/distutils-builbot/ eggs/tmptMBdZk" "-q" "/Users/ssteiner/Dropbox/src/twisted_trunk" path=/Users/ssteiner/Dropbox/work/distutils-builbot/eggs/ setuptools-0.6c9-py2.6.egg /Library/Python/2.6/site-packages/Pyrex-0.9.8.5-py2.6.egg/Pyrex/ Compiler/Errors.py:17: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 self.message = message unrecognized .svn/entries format in conftest.c:1:23: error: sys/epoll.h: No such file or directory conftest.c:1:23: error: sys/epoll.h: No such file or directory conftest.c:1:23: error: sys/epoll.h: No such file or directory lipo: can't open input file: /var/folders/28/28Xggq0cGXqLpT1Uz5Gk4E++ +TI/-Tmp-//ccOmCair.out (No such file or directory) Got Twisted 8.2.0-r27376. Picked: Twisted = 8.2.0-r27376 Getting required 'zope.interface' required by collective.buildbot 0.3.5. required by Twisted 8.2.0-r27376. ... and then it carries on. However, when trying to run it with /bin/test, something tries to refer to "trunk-checkout" when the python trunk was actually checked out into python-trunk. (~/work/distutils-builbot)# bin/test Traceback (most recent call last): File "bin/test", line 12, in <module> os.chdir(python_trunk) OSError: [Errno 2] No such file or directory: '/Users/ssteiner/work/ distutils-builbot/parts/trunk-checkout/python' All in all, I really can't get it to build or run anywhere at this point... S
On Mon, Oct 19, 2009 at 4:35 AM, Steve Steiner <ssteiner@mac.com> wrote:
sounds like version problems.
This is running on a bare system, after figuring out which development tools were needed, all the version control systems necessary etc. I have all that captured in my Fabric script so we're all set for being able to duplicate it.
I have NOT installed anything into the system Python at all. In fact, other than the tools necessary to pull the products out of VCS, and the compilers and such, I haven't installed (or done) anything to the system at all. That's kind of the point. We want to be able to go from bare metal to running buildbot completely automatically.
I tried to build the buildbot again on debian and Mac OSX and it worked fine Didn't try on Ubuntu though. It seems that it's using the buildbot installed in your system, which might be the problem since the buildout doesn't pinpoint the versions. Here, I use Python 2.6 and I have: - buildbot-0.7.11p3 - Twisted-8.2.0 - collective.buildbot-0.3.5 I suggest that you install virtualenv and that your run this sequence in an Ubuntu that has all the development tools we have discussed (python-dev, zlib, etc) : $ cd /var/ $ mkdir distutils-bot $ cd distutils-bot $ virtualenv --no-site-packages . $ hg clone https://tarek@bitbucket.org/tarek/distutils-builbot . $ bin/python bootstrap.py $ bin/buildout -c buildbot.cfg This is supposed to compile. Last option: give me an ssh access to your ubuntu box if possible, I'll probaby be able to fix it. Tarek
On Mon, Oct 19, 2009 at 9:34 AM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
I tried to build the buildbot again on debian and Mac OSX and it worked fine Didn't try on Ubuntu though.
Here's this very buildbot running on debian: http://ziade.org:9080/waterfall
On Oct 19, 2009, at 3:36 AM, Tarek Ziadé wrote:
On Mon, Oct 19, 2009 at 9:34 AM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
I tried to build the buildbot again on debian and Mac OSX and it worked fine Didn't try on Ubuntu though.
Here's this very buildbot running on debian:
I'm not disputing that it runs, I'm telling you that it won't run on a clean Ubuntu without some additional setup. S
On Oct 19, 2009, at 3:34 AM, Tarek Ziadé wrote:
On Mon, Oct 19, 2009 at 4:35 AM, Steve Steiner <ssteiner@mac.com> wrote:
sounds like version problems.
This is running on a bare system, after figuring out which development tools were needed, all the version control systems necessary etc. I have all that captured in my Fabric script so we're all set for being able to duplicate it.
I tried to build the buildbot again on debian and Mac OSX and it worked fine Didn't try on Ubuntu though.
It seems that it's using the buildbot installed in your system, which might be the problem since the buildout doesn't pinpoint the versions.
Here, I use Python 2.6 and I have: - buildbot-0.7.11p3 - Twisted-8.2.0 - collective.buildbot-0.3.5
I suggest that you install virtualenv and that your run this sequence in an Ubuntu that has all the development tools we have discussed (python-dev, zlib, etc) :
The whole idea here is to make a setup script that sets the machine: Fire up a new cloud server Install whatever software is necessary Download the buildbot code Build the buildbot Start it running for as long as needed Kill the server We pay for the server by the hour (1.5 cents/hr) and can fire up as many as necessary. I will put my fabric script up on bitbucket in my clone and you can pull it over and see what I mean. Right now, I"m just using a local VM, but the idea would be to put a fresh one up on Rackspace as needed, then kill when done.
$ cd /var/ $ mkdir distutils-bot $ cd distutils-bot $ virtualenv --no-site-packages . $ hg clone https://tarek@bitbucket.org/tarek/distutils-builbot . $ bin/python bootstrap.py $ bin/buildout -c buildbot.cfg
This is supposed to compile.
Last option: give me an ssh access to your ubuntu box if possible, I'll probaby be able to fix it.
I am not looking to hand-fix this, that would be pointless. I want an automated build script that will fire up a brand new Ubuntu cloud server, set up the entire environment, install the buildbot and run. S
On Mon, Oct 19, 2009 at 1:44 PM, Steve Steiner <ssteiner@mac.com> wrote:
The whole idea here is to make a setup script that sets the machine:
Fire up a new cloud server Install whatever software is necessary Download the buildbot code Build the buildbot Start it running for as long as needed Kill the server
We pay for the server by the hour (1.5 cents/hr) and can fire up as many as necessary.
I will put my fabric script up on bitbucket in my clone and you can pull it over and see what I mean.
Right now, I"m just using a local VM, but the idea would be to put a fresh one up on Rackspace as needed, then kill when done.
Sounds good.
$ cd /var/ $ mkdir distutils-bot $ cd distutils-bot $ virtualenv --no-site-packages . $ hg clone https://tarek@bitbucket.org/tarek/distutils-builbot . $ bin/python bootstrap.py $ bin/buildout -c buildbot.cfg
This is supposed to compile.
Last option: give me an ssh access to your ubuntu box if possible, I'll probaby be able to fix it.
I am not looking to hand-fix this, that would be pointless. I want an automated build script that will fire up a brand new Ubuntu cloud server, set up the entire environment, install the buildbot and run.
By "fix it" I mean "debugging it in your environment to find the problem" of course. Anyways, let see what my sequence does in your VM (with python-dev installed first, and zlib) And if it fails, ls me the content of eggs/ as well Tarek
On Mon, 19 Oct 2009 07:44:56 -0400, Steve Steiner <ssteiner@mac.com> wrote:
I am not looking to hand-fix this, that would be pointless. I want an automated build script that will fire up a brand new Ubuntu cloud server, set up the entire environment, install the buildbot and run.
Can I help? It's actually a pretty interesting and typical problem. One of the big challenges is spanning the internal reach of python packaging to 'the outside'. I wish I could use a mac - where we don't have these problems. But under Linux, even on a good distro like debian or ubuntu, it can take a while to locate all the compilers and external tools that are required to build your given package. Doing 'fresh' installs on a fresh os surely is a typical developer use case... or should be going forward.. It should be pretty simple to specify external os-packages and pull them in. Somehow.. :-) (apt-get) There must be some way to put these into a coherent config file. David
On Oct 19, 2009, at 8:57 AM, David Lyon wrote:
On Mon, 19 Oct 2009 07:44:56 -0400, Steve Steiner <ssteiner@mac.com> wrote:
I am not looking to hand-fix this, that would be pointless. I want an automated build script that will fire up a brand new Ubuntu cloud server, set up the entire environment, install the buildbot and run.
Can I help?
I've just put the fabfile up in the distutils-buildbot http://bitbucket.org/tarek/distutils-buildbot/overview/ Feel free to jump in! I've run that exact fabfile onto a pure fresh ubuntu 9.04 setup and it won't run yet. Any insight you can provide would be much appreciated. I'd love to make a "make me a buildbot" tool; I think it would be incredibly useful, generally. S
On Mon, Oct 19, 2009 at 3:23 PM, Steve Steiner <ssteiner@mac.com> wrote:
On Oct 19, 2009, at 8:57 AM, David Lyon wrote:
On Mon, 19 Oct 2009 07:44:56 -0400, Steve Steiner <ssteiner@mac.com> wrote:
I am not looking to hand-fix this, that would be pointless. I want an automated build script that will fire up a brand new Ubuntu cloud server, set up the entire environment, install the buildbot and run.
Can I help?
I've just put the fabfile up in the distutils-buildbot http://bitbucket.org/tarek/distutils-buildbot/overview/
Feel free to jump in!
I've run that exact fabfile onto a pure fresh ubuntu 9.04 setup and it won't run yet.
Any insight you can provide would be much appreciated.
I'd love to make a "make me a buildbot" tool; I think it would be incredibly useful, generally.
Beware that for the server part of buildbot, it doesn't work on Windows, Maybe this has changed but that's the last status I know of. But I definitely want to run the slaves on Win32. This would require the buildout to make the distinction between a slave machine and a machine that has a salve and a master. It' not the case right now but I can change it, by adding a 'only_slave.cfg' file Tarek
participants (3)
-
David Lyon -
Steve Steiner -
Tarek Ziadé