[Pythonmac-SIG] Python development on OSX

[Ned Deily]

In article 
<AANLkTinYTR56WGPjgh1vyxBSMBxV8tRVp4ZjWLHyZzy2 at mail.gmail.com>,
>    1. Why are there so many "variations" of the Python installer for the
> Mac? (I'm not referring to 32-bit vs 64-bit architectures, although that
> does complicate things quite a bit).
> 
> For example, there is the OSX installer at Python.org, then there are the
> Macports python variations, and I recently read about something called
> Macpython.
> 
> How does one know which o to use?

You left out a few!  Why are there so many?  History, varying needs, 
release schedules, because it's a fun thing to do - those are some of 
the reasons.  Here is my take on it.

Since at least Mac OS X 10.3 (7+ years ago), Apple has been including 
one or more versions of Python as part of OS X and over the years 
they've added more and more third-party Python packages and a few of 
their own Python add-ons.  For many people's needs, the Apple versions 
are sufficient.  On the other hand, like many operating system 
distributors, Apple tends to be very conservative about which versions 
it includes in major OS X releases, which tend to come out every two 
years or so, and they usually only supply security and some bug fixes in 
between major releases. For example, the current OS X release, 10.6.6, 
has Python 2.6.1 (released 2008-12) and 2.5.4 built in.  At the moment, 
Python 2.7.1 and Python 3.1.3 are current.  So, people looking for the 
latest and greatest Python releases usually have to look elsewhere.

Back in the old days prior to Mac OS X, there was a project to make 
Python available on its predecessor, Mac OS 9 (and earlier).  That was 
known as MacPython and, in the early releases of Mac OS X, it continued 
to make releases for OS X.  At some point, the MacPython project was 
merged into the standard Python Software Foundation (PSF or python.org) 
development and release process and installers for OS X begin being 
supplied as part of "official" releases of Python, rather than from a 
separate MacPython project.  That's where we are today although there 
are still vestigial references to MacPython online.

One of the interesting characteristics of Mac OS X is that it combines 
technology from different origins and so it presents different faces to 
different kinds of users.  A lot of stuff comes from its NeXT operating 
system heritage, a somewhat non-traditional Unix.  When Apple decided to 
use it as the basis for Mac OS X, features were added to make the 
transition easier for users of traditional Mac OS.  And more and more 
vanilla open-source products and features from modern Unix-y systems 
(like the BSDs and Linux) were incorporated.  That shows up in Python in 
a few ways, including two rather different ways of installing Python on 
OS X: a traditional shared library approach as is used on nearly all 
other Unix-style systems, and a unique-to-OSX "framework" install which 
is based on NeXT practices.  Most distributors of OS X Pythons use the 
latter but not all.

Another feature of OS X, rarely seen elsewhere, is the ability to have 
executable code for multiple architectures "glued" together into one 
file, a so-called universal file.  That allows Apple to distribute one 
set of executable files and libraries that can support current and 
previous architectures. That was used to ease the big transition from 
PPC to Intel CPUs and, more recently, from 32-bit Intel to 64-bit Intel.  
It allows for a lot of flexibility and backwards compatibility but at 
the cost of added complexity for third-party languages like Python.  So 
some of the reasons for different Python installers have been because of 
what architectures are supported and differing approaches to backwards 
compatibility (like how many OS X releases to support with one version).

Another reason: while Apple supplies a lot of open source software as 
part of OS X, it doesn't supply everything.  One common reason is 
because of licensing issues (avoiding GPL libraries, for example).  
Also, Apple does not supply a package manager for its software, like 
those developed over the years for other open source systems.  There 
have been at least three projects that have evolved to address that: 
Fink which provides many of the Debian/Ubuntu packages for OS X (without 
the Linux kernel of course); MacPorts (previously known as DarwinPorts) 
which provides the FreeBSD package ports; and, more recently, HomeBrew 
which has its own way of managing packages.  All three provide Python 
ports and the associated infrastructure.  And that can be very useful on 
Mac OS X when you want to install and run Python packages that depend on 
third-party libraries not included with OS X.  Classic examples of those 
are the Python Imaging Library (PIL) and MySQLdb.  Both require libs not 
included in OS X. You can build everything yourself but, for everything 
to work, you need to make sure that all the pieces are compatible, that 
is, built with compatible archs and deployment targets.  A good package 
manager, like MacPorts or Fink, will eliminate the guess work and 
automatically build and install all the required dependencies in a 
compatible way so that it should all just work.  That's probably the 
biggest reason for the popularity of installing Python from one of those 
distributors: it can be a huge time saver.

On the other hand, it is very difficult for volunteer-staffed projects, 
like most of these are, to keep on top of every package and keep them 
all up-to-date with optimal configuration selections.  That's one of the 
reasons for commercial distributions, like ActiveState Python and the 
Enthought Python distribution, both of which build off of python.org 
configurations but add their own packages and offer paid support options 
on top of things.  They also may target specific communities: EPD caters 
to the scientific / numeric user community.

Another complicating factor has been the way third-party Python packages 
have been made available.  Over the past ten years are so, there have 
been a number of key improvements and growing standardization in Python 
packaging, first with Distutils and then later easy_install (setuptools) 
and now Distribute, Pip, and buildout to name a few.  Not so long ago, 
it was much more difficult for users to install more complex packages 
especially those with C extension modules and so some projects 
distributed their own binary installers for various platforms including 
dependent libraries, like you've seen with wxPython.  These days it is a 
lot easier to automatically build and install Python packages, however 
the problems associated with dependent libraries often remain (which is 
a strength of the MacPorts or Fink comprehensive solution approach).

So, all in all, there are lots of reasons plus others I'm forgetting.  
Many of them are not unique to the Mac world.  But that doesn't make 
things less daunting when you first dive in.

-- 
 Ned Deily,
 nad at acm.org