[Python-ideas] CPython needs to revamp/replace it's build system && Cross-compiling Python sucks

[Nick Coghlan]

On 10 March 2015 at 21:20, Russell Keith-Magee <russell at keith-magee.com> wrote:
>
> On Tue, Mar 10, 2015 at 10:43 AM, Ryan <rymg19 at gmail.com> wrote:
>>
>> Russell Keith-Magee <russell at keith-magee.com> wrote:
>> >Hi Ryan,
>> >
>> >First off I'm no fan of autoconf, but...
>> >
>>
>> Who is other than the one who designed it? :)
>
> Let me be clear - there's a difference between "not a fan" and "calling for
> it's head". Autoconf is crufty, annoying, and sometimes aggravating - but it
> *works*, and has had some pretty seriously battle testing - both in the
> abstract "autoconf as a product", and in the specific "autoconf as used in
> Python". This isn't something that can be said of most of the alternatives
> in the abstract case, or *any* of the alternatives in the Python case.

Building architecture dependent software is a genuinely hard problem,
which is why it's full of "80%" solutions like autotools and the
setuptools/distutils pairing. Yes, they're quirky and annoying, but
they also basically do their job, so folks tend to develop a high
tolerance level for their quirks.

In our particular case, we also have the fun problem of "How do you
build CPython locally when you don't have a working Python interpreter
yet?". A number of the more interesting alternative C/C++ build
systems (like Meson, Scons, Waf and the Google "gyp" project that
powers the node-gyp native extension build system for Node.js) require
an already working Python interpreter to get things started (when
starting from a completely blank slate, even our own build sequence
has to build a deliberately broken partial interpreter just to freeze
importlib._bootstrap before going on to build a properly configured
one that can actually import modules).

CMake seems like a plausible alternative that would avoid the
bootstrapping problem (as even building it from source only requires a
working C++ compiler for the platform, either native or
cross-compiled): http://www.cmake.org/

KDE switched over to CMake from autotools ~9 years ago
(http://lwn.net/Articles/188693/) and plenty of other projects have
either followed suit or simply never used autotools in the first
place: http://www.cmake.org/Wiki/CMake_Projects

It's far too late in the 3.5 dev cycle to contemplate a change like
that, but if anyone was looking for a potentially interesting side
project, seeing exactly what was involved in getting CPython to build
with CMake instead of autotools could be a worthwhile investigation:
https://stackoverflow.com/questions/7132862/tutorial-for-converting-autotools-to-cmake

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia