[Python-Dev] Automatic installer builds (was Re: Fwd: Broken link to download (Mac OS X))

[David Bolen]

"Martin v. Löwis" <martin at v.loewis.de> writes:

> Not sure what you mean by "make available" - I thought this is just a
> matter of configure options?

Building as a framework, yes.  But I think there's some steps to take
to then have the test python binary use the locally built framework
while running the tests (since the framework won't be in a system
location like /Library/Frameworks during the test). Probably similar to
whatever py2app does to use packaged frameworks contained within the
application bundle.

I doubt it's overly complex, but just an extra step that might need to
be incorporated into any buildbot test rules or script to ensure the
just built framework is used in the tests.

> Would you like to see such a build on your machine in addition, or
> instead of, the Unix build?

I suppose an argument could be made to test both, since others have
indicated here they often do a normal Unix-oriented build on OSX, but
I think if I only had to pick one, I'd go with framework to match what
gets published as a binary for downloading.  Theoretically it should
probably also be a universal build, though of course only one variant
would actually get tested on a build slave depending on its platform.

Certainly anything we could do to make the buildbot generated builds
match as closely to possible the distribution binary process would be
beneficial, I'd think.

>> On Windows, the buildbot tasks use scripts that keep the third party
>> stuff in the buildbot branch tree itself.  I wonder if perhaps with a
>> little TLC, we could come up with some Mac-specific buildbot scripts
>> to better have an OSX build slave mimic the final build process.
>
> No. I'd rather create a separate builder on the master.

I'm wondering if I'm saying the same thing but just not using the
right vernacular.  Something on the master (a builder?) for Windows
instructs it to run Tools\buildbot\build.bat for building and test.bat
for testing, rather than each of the individual commands.

Whether we encapsulate the needed logic in separate makefile targets
for the Mac, or independent build scripts like on Windows, the build
master would only need to execute some Mac appropriate command on a
given branch (not sure if that's separate builders), with the
makefile/script in the given checkout handling the third party stuff
and environmental setup.

Keeping the knowledge in the makefile or script in the source tree would
let the rules change across branches without affecting the build master.
Though if having more specific rules in the master was easier, I'd be
fine with that too.

> It would be possible to commit these packages into the externals
> repository, just as we do for Windows. It would then be possible to
> check them out over again all the time, or somehow to detect when the
> URL changes so they export a different subversion directory.

Sure - that's the sort of thing I figured a build script could take
care of, much as it does on Windows.  Or, the current Mac
build-installer script already has the necessary information, and can
be instructed where to place the third party stuff, so it might only
be necessary to have the buildbot process run that script with
appropriate build tree-relative paths.

> For 2.7, I would do that only if the very same build process is also
> going to be used for the final release. There is no point in testing
> builds when then the final release uses some other process.

Agreed, thus my caveat as to the output of the build-installer script
in fact being what has been published for downloads.  In my brief
tests it looks like it created a production DMG ready for publishing
on the download page, and did in fact install correctly, but I'm still
new to the Mac binary building process.

>                                                             It would
> also be good if anybody who commits to producing OSX binaries now would
> also produce them throughout the 2.7 lifetime (which could be a bit
> longer than the traditional 18 months).

To the extent that the installer script is in fact the definitive
process, and if the machine (10.4 Intel) is considered suitable as a
binary build platform, then worst case we could probably have the
buildbot run the script and upload the result when needed (sort of a
one-shot version of the old MSI daily builder).  I don't plan on the
machine going away any time soon.

Of course, this is just the DMG construction.  Not sure what amount of
"test the installer" testing should be required prior to publication.

-- David