[Python-Dev] Automatic installer builds (was Re: Fwd: Broken link to download (Mac OS X))

[Steve Holden]

David Bolen wrote:
> "Martin v. Löwis" <martin at v.loewis.de> writes:
> 
>>                      This actually happened on Windows - some people now
>> recommend to run the buildbot scripts on a regular developer checkout,
>> because they supposedly do the right things.
> 
> I have to admit that I'm guilty of this (though to be fair most of my
> test builds are buildbot-related), if only because it takes care of
> all the external stuff automatically and self-contained in the build
> tree.
> 
>> I would rather prefer to have the buildbot run the same commands that we
>> recommend developers to run. The knowledge about these should be in the
>> README, and it should be stable knowledge, i.e. require infrequent
>> changes. This is true for Unix: configure/make/make test/make clean
>> had been the correct procedure for ten years now. The Unix buildbot only
>> deviates slightly, to have the slaves run a more expensive version of
>> the test suite.
> 
> In digging around a bit, it would appear that there's actually quite a
> bit of OSX support already in the Makefile (either the main one or
> the one in Mac).  There's even a target that tests both halves of a
> universal build (using rosetta for the PPC version) on an Intel box.
> 
> I suspect it's just a question of setting up a Mac-appropriate
> builder, using the universal SDK in the configure step, and whatever
> Makefile targets are deemed best and current, perhaps with the
> addition of support for pulling in the third party stuff through
> externals or whatever.  A first pass could just be to factor that
> process into a separate stage of build-installer that could be run
> independently of the rest of the installer build process.
> 
> In the interim, I've generated the third-party libraries via the
> current trunk build-installer script and installed them in /usr/local
> on my buildbot, so they are found by any of the buildbot builds using
> the stock configure/make approach.  Other than a slight update to the
> bzip version, looks like the dependency versions in the installer
> script haven't changed for over a year, so I suspect the libraries are
> fine for any of the branches currently being built.
> 
> I also updated to the latest 8.4.19 Tcl/Tk in /Library/Frameworks
> since I saw some interpreter crashes in tests in what appeared to be a
> Tcl code path.  It had been building against my system 8.4.7 Tcl.  The
> Windows buildbot uses Tcl 8.5 - not sure if that should be preferred
> for the Mac buildbot as well, but will leave it at 8.4 for now.
> 
> I think this should create test builds more representative of the
> final binaries.  It's not testing a universal framework build, but the
> test target only tests the Intel path anyway, the generated code
> should still be the same, and the same libraries are in use.
> 
> If anyone more familiar with this side of things for OSX has some
> spare time down the road, I'd be happy to help work on improving the
> process for the buildbot.
> 
>> I'd be interested in setting up daily builds then. For these, I think
>> it's fine that they may differ slightly from the official binaries -
>> people would recognize that they are testing a different set of binaries.
> 
> We can certainly start by just trying to automate the execution of
> build-installer, something I suspect can be completely controlled from
> the master side, and then uploading the resulting dmg file.  I'd be
> happy to help coordinate any experiments offline if you're interested.
> 
> I do currently have a DMG built for 2.7 Beta 1, if it would be useful.
> 
Great work David, this is a terrific step forward. Thanks!

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
See PyCon Talks from Atlanta 2010  http://pycon.blip.tv/
Holden Web LLC                 http://www.holdenweb.com/
UPCOMING EVENTS:        http://holdenweb.eventbrite.com/