[Python-Dev] OS X Installer for 3.0.1 and supported versions

[Ned Deily]

Speaking of an OS X installer for 3.0.1, over the last few weeks I have 
been working on tidying up the OS X installer build process.  While the 
basic OS X build/installer process is good, some cruft has accumulated 
over the past years and a number of mostly minor issues arose due to the 
3.x split.  IMO, the most important issues were with IDLE and, thanks to 
Ronald, we did get the most important fixes for OS X IDLE checked-in in 
time for 3.0.1; they are also in py3k and will be going into trunk and 
26.  I have a few other fixes that apply just to the OSX build/installer 
parts which did not get submitted in time for the 3.0.1 cutoff but which 
are ready to go for 3.x and 2.x.  Basically they fix some version number 
updating and ensure that the installer image will be built reproducibly 
in a clean environment so there is no contamination of the installer 
images.  Currently, that's easy to do as happened with the first round 
of the OS X 2.6 installer (e.g. with a locally installed Tcl/Tk).  They 
also allow images to build for various OS X version.  More on that below.

I would like to get to the point where the OS X images can be generated 
and tested automatically.  I think that is reasonable and achievable.  
It's not quite there yet but I can now semi-automatically produce images 
for 2.6, 2.7 (trunk), 3.0, and 3.1 (py3k) in both the traditional "fat" 
(32-bit i386) for 10.4/10.5 and, for 10.5, "universal" (4-way 32-bit and 
64-bit intel/ppc).  They all have been passing the standard regrtest 
with no new obvious (to me) regressions but I certainly won't claim to 
have done complete and thorough testing.  (In particular, I have no 
access to a G5 for 64-bit PPC testing.)

However the "official" OS X images are built, there is an important 
issue about them that should be addressed now.   That issue is which OS 
X versions should be supported by installer images.  (This may well have 
been discussed before so my apologies if I am covering old ground here.)

Some background: OS X 10.4 (Tiger) introduced 2-way ("fat") executables 
and libraries which include object code for both ppc and i386 in the 
same files.  OS X 10.5 (Leopard) extended that support to 4-way 
("universal"), adding 64-bit x86_64 and ppc64.  In 10.5, much, but not 
all, of OS X and a growing number of applications are shipped as 4-way 
universal.  The OS X developer tools include SDKs to allow cross-builds 
for various OS X levels and architectures.  There is a build-time 
environ variable, MACOSX_DEPLOYMENT_TARGET, which specifies the minimum 
OSX releases that an executable will work with. 

Starting in 2006, the 2.5.x series of Python OS X installers were 
released as "fat" but built with MACOSX_DEPLOYMENT_TARGET=10.3 so that 
the same installer image would work on 10.3 PPC and 10.4 PPC and intel.   
Unfortunately, along the way, there was a significant bug in 
distutils.util.get_platform not returning "fat" in the platform string; 
among other things this has had impact on packages on PyPI.  Ronald 
fixed the problem prior to 2.5.2 (?) but, for compatibility, the fix is 
only effective if MACOSX_DEPLOYMENT_TARGET >= 10.4 and the 2.5.x 
installers continued to be built as 10.3.  Also unfortunately, the 
deployment target for the first 2.6 OS X installer was not updated.  In 
the meantime, Ronald added support for 64-bit and 4-way universal builds 
for 10.5 but, to date, no official installer images have been released 
with that enabled although clearly people are already building their own 
64-bit OS X pythons.

The last Apple point release of 10.3 was in 4/2005.  10.4 was also 
released then.  The last Apple maintenance release of 10.4 was in 
11/2007.  10.5 was released in 10/2007 and, with subsequent update 
releases, remains the current system.  While no release dates have been 
announced, 10.6 (Snow Leopard) is rumored to be nearing release.  If 
Apple follows past practices, they will likely terminate all support for 
10.4 (release n-2) when 10.6 releases and will continue to support 10.5 
(n-1) for the lifetime of 10.6.  Needless to say, Apple stopped 
supporting 10.3 a long time ago and, if 10.6 does release in the not 
too-distant future, 10.4's days are numbered.

What does this mean for Python?  To date, there have been no official 
3.0 OS X installers released.  Because of the deficiencies of building 
for the long-unsupported 10.3 (the distutils bug, the lack of 64-bit 
support, and probably other things I'm not aware of), I think it is very 
important that the 3.0.1 get off on the right foot by changing the 
minimum supported versions now.

I see three plausible options:

1. Release an installer built for 10.5 and higher.
   pros: delivers 32-support and 64-support;
   cons: prematurely disenfranchises 10.4 users

2. Release an installer built for 10.4 and higher.
   pros: one size fits all
   cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc

3. Release two installers, one each for 10.4+ and 10.5+.
    pros: supports current and future systems;
           delivers 64-support to 10.5+ users;
           could choose to drop 10.4 installers anytime after 10.6 
           releases;
    cons: some extra work to build/release
             (but not much and not often);
             others??

Note, this is only talking about the python.org installer images, not 
Python itself.  Users of older systems (10.3 and earlier) would still be 
able to build their own Pythons or even installers.  However, there 
could (should?) be an explicit statement made as to what OS X versions 
are "officially" supported.

With the improvements to the build process and the experience I've had 
so far, I would strongly recommend option 3.  I am willing to commit to 
supplying those images for 3.0.1 and beyond until we can get to a more 
automated process.  Or I'd be happy to work with someone else to make it 
happen for 3.0.1.  Now that the tarball is out, I will be able to get 
the images built and tested this weekend.

BTW, over on the pythonmac forum, there has been discussion of having 
some Mac activities at Pycon.  Ronald is planning to be there and I'm 
hoping to, as well, so that could be a great opportunity to discuss and 
work on futures.  And this same discussion and decision needs to be made 
going forward for 2.7 and 2.6.x (I think the change should be made for 
2.6.2).

So, to summarize, I will build and test installer images for 3.0.1 built 
for 10.4+ and for 10.5+ and will make them available, if there is 
interest.  If someone else is planning on making official installer(s), 
I'd be happy to pass along my experiences so far.  In any case, I 
strongly urge that the minimum support level be 10.4.

--Ned

-- 
 Ned Deily,
 nad at acm.org