Python Extension Building Network

[Yu-Xi Lim]

kyosohma at gmail.com wrote:
> Hi,
> 
> I am trying to get a small group of volunteers together to create
> Windows binaries for any Python extension developer that needs them,
> much like the package/extension builders who volunteer their time to
> create Linux RPMs.
> 
> Mike
> 

It's not entirely clear to me what you're compiling. Most of the modules 
you have there do not have native code. I'm guessing your main focus is 
building .exe installers for Windows users.

If it's just installers of pure Python code, why the extra effort of 
using both VS and MingW? AFAIK, they both yield functionally identical 
results---the exact same .py files get installed.

I'm also not sure how big a task this is or the issues involved, but 
here's my take based on what I've read:

Throwing more manpower at the task is not the solution. The actual 
process can be and SHOULD BE highly automated. All that's needed are a 
proper distutils script, and proper tests. Both responsibilities should 
fall on the shoulders of the module authors, though I guess experienced 
volunteers can help out with the former.

If the module is pure Python, there should be little need for testing 
the installer specifically on Windows. If there's a failure, then the 
module itself is buggy (e.g. making platform-specific assumptions) or 
there's similar problem with the install script.

For a large number of modules on PyPI, building the installer is trivial 
(assuming pure Python, and a setup script). Compiling them can be easily 
automated: 1) check PyPI for updates 2) download the latest update 3) 
build the .exe 3) upload to a website. All that remains is the need for 
some hardware (a build farm) and the occasional manual intervention.

If I'm not mistaken, distutils can build Windows installers on other 
platforms. Maybe you could set up a Linux LiveCD with the necessary 
tools installed and distribute that. Module authors who want to build 
Windows installers can easily use that CD. Volunteers wanting to help 
can do so easily without having to repeat the task of setting up the 
toolchain separately.


I've done some work as a port maintainer (aka package maintainer) for 
FreeBSD. The BSD port system is somewhat closer to PEAK's Easy Install, 
however binary packages (similar to RPMs or DEBs) are also available. 
The port maintainer's job is solely to ensure that the Makefile (build 
script) is working and up to date. The actual testing of the code is 
done by the authors of the software itself and the port maintainer only 
ensures that the install script works right. Binary packages are built 
by an automated system using a cluster of computers for various 
architectures and supported OS versions. Errors during builds are sent 
to to port maintainers.

A similar system is used for most Linux distributions with a central 
repository for packages.