[Python-Dev] Automatic installer builds (was Re: Fwd: Broken link to download (Mac OS X))

[David Bolen]

"Martin v. Löwis" <martin at v.loewis.de> writes:

>                      This actually happened on Windows - some people now
> recommend to run the buildbot scripts on a regular developer checkout,
> because they supposedly do the right things.

I have to admit that I'm guilty of this (though to be fair most of my
test builds are buildbot-related), if only because it takes care of
all the external stuff automatically and self-contained in the build
tree.

> I would rather prefer to have the buildbot run the same commands that we
> recommend developers to run. The knowledge about these should be in the
> README, and it should be stable knowledge, i.e. require infrequent
> changes. This is true for Unix: configure/make/make test/make clean
> had been the correct procedure for ten years now. The Unix buildbot only
> deviates slightly, to have the slaves run a more expensive version of
> the test suite.

In digging around a bit, it would appear that there's actually quite a
bit of OSX support already in the Makefile (either the main one or
the one in Mac).  There's even a target that tests both halves of a
universal build (using rosetta for the PPC version) on an Intel box.

I suspect it's just a question of setting up a Mac-appropriate
builder, using the universal SDK in the configure step, and whatever
Makefile targets are deemed best and current, perhaps with the
addition of support for pulling in the third party stuff through
externals or whatever.  A first pass could just be to factor that
process into a separate stage of build-installer that could be run
independently of the rest of the installer build process.

In the interim, I've generated the third-party libraries via the
current trunk build-installer script and installed them in /usr/local
on my buildbot, so they are found by any of the buildbot builds using
the stock configure/make approach.  Other than a slight update to the
bzip version, looks like the dependency versions in the installer
script haven't changed for over a year, so I suspect the libraries are
fine for any of the branches currently being built.

I also updated to the latest 8.4.19 Tcl/Tk in /Library/Frameworks
since I saw some interpreter crashes in tests in what appeared to be a
Tcl code path.  It had been building against my system 8.4.7 Tcl.  The
Windows buildbot uses Tcl 8.5 - not sure if that should be preferred
for the Mac buildbot as well, but will leave it at 8.4 for now.

I think this should create test builds more representative of the
final binaries.  It's not testing a universal framework build, but the
test target only tests the Intel path anyway, the generated code
should still be the same, and the same libraries are in use.

If anyone more familiar with this side of things for OSX has some
spare time down the road, I'd be happy to help work on improving the
process for the buildbot.

> I'd be interested in setting up daily builds then. For these, I think
> it's fine that they may differ slightly from the official binaries -
> people would recognize that they are testing a different set of binaries.

We can certainly start by just trying to automate the execution of
build-installer, something I suspect can be completely controlled from
the master side, and then uploading the resulting dmg file.  I'd be
happy to help coordinate any experiments offline if you're interested.

I do currently have a DMG built for 2.7 Beta 1, if it would be useful.

-- David