[New-bugs-announce] [issue3871] cross building python for mingw32 with distutils

Roumen Petrov report at bugs.python.org
Mon Sep 15 01:08:19 CEST 2008

New submission from Roumen Petrov <bugtrack at roumenpetrov.info>:

This is a completely new patch against trunk that try to resolve mingw32
build. The first version to show problems with current python builds
system, to propose solutions/work-arounds and to be stating point for
discussion. Also I prefer some issues to be resolved later when we agree
on the basic issues.

The patch is tested in cross-compilation environment and native build
has to be tested too. Also the patch show how to use cross-compilation
environment and require patch posted in
issue 3754 ("minimal cross-compilation support for configure") to be
applied first.

The new patch share many commons with issue 841454 "Cross building
python for mingw32" posted by Andreas Ames, on 2003-11-13), but build
process use distutils instead scons.
Also some mingw related changes from issue 1597850 ("Cross compiling
patches for MINGW" - Han-Wen Nienhuys, from 2006-11-16) are shared too,
but we differ in concept how to use cross-compilation environment.
The correct way is to use $host variable (see issue 3754).

Also my patch use python installed on build system along with some hacks
to overcome distutils limitations.

The main issue "how to select correct compiler" in cross-compilation
environment isn't resolved at all. The patch is tested with installed
python version 2.4. 
Usually the cross-compiler is installed in same directories as native
compiler but tools are prefixed by <CPU>-<VENDOR>- (see meanign of "host
triplet"). In this environment we can't cross-compile even if we select
mingw as compiler in arguments for "setup.py":
The method get_versions() from cygwinccompiler.py call find_executable
for to find gcc, ld, dllwrap but on build system they point to native
compiler and that aren't prefixed. Also the expression "result =
re.search('(\d+\.\d+(\.\d+)*)',out_string)" return as example
'' and later StrictVersion fail since value is not in format
N.N{.N{A{N}}}. If we replace '*' with '?" in regular expression we will
get correct  result, but this isn't right solution.
So the question is 'how to pass compiler to distutils' (as example for
python 2.4) remain open.

Work-around: install mingw cross-suite in own directory (MINGW_ROOT) and
prepend PATH environment variable with both paths:
- $MINGW_ROOT/<MINGW_HOST>/bin where <MINGW_HOST> is something like
i586-pc-mingw32 (depend from you host selected when you build suite).
In this case find_executable will find mingw ld and since -v option
return something like "GNU ld version 2.17.50 20060824", regular
expresion value that make StrictVersion happy.

Note that in both environment configure script find compiler tools
prefixed as example by "i586-pc-mingw32-".
My mingw is build for host "i386-mingw32msvc"(this isn't correct, but is
acceptable for host. For next version I plan to use for build the
correct one, i.e i386-pc-mingw32). So in the final Makefile I see:
CC=             i386-mingw32msvc-gcc
AR=             i386-mingw32msvc-ar
RANLIB=         i386-mingw32msvc-ranlib

Also setup.py set compiler attribute linker_so - for details see
comments in the patch.

Another point is to link installed python in top-build directory so that
on posix system distutils read variables from our makefile instead from
installed in build system. In this case python_build (an internal to
distutils flag) is set.

A, but not common, build issue is that some python 2.4 versions will
search for pyconfig-32.h instead pyconfig.h.
running build
running build_ext
error: ./pyconfig-32.h: No such file or directory
I see this on cent-os (5.0?), but I can't see on slackware 11.0.
This patch don't address this.
Just go in top-build directory and create a link, i.e. "ln -s
./pyconfig.h pyconfig-32.h".

If you system support emulation after build you may run python.
The build python search for modules. After patch modules are with suffix
same as for native build - .pyd. You may thin them to the top-build
directory or to follow binary distribution - create directory DLLs,
enter into it and link pyd-files:
$ for f in ../build/lib.*/*.pyd; do ln -sf $f; done

Differences between mingw build python and binary distribution(native
Mingw build follow posix rules and will create more modules than native
build. In the native build they are part from main executable(buildin).
This is the list:
array, audioop, binascii, _bisect, _bytesio, cmath, _codecs_cn,
_codecs_hk, _codecs_iso2022, _codecs_jp, _codecs_kr, _codecs_tw,
_collections, cPickle, cStringIO, _csv, datetime, _fileio, _functools,
future_builtins, _heapq, _hotshot, imageop, itertools, _json, _locale,
_lsprof, math, mmap, msvcrt, _multibytecodec, operator, parser, _random,
strop, _struct, _subprocess, time, _weakref, _winreg, zlib.
I think that this inconsistency is problem of native build and for now I
don't address it.

Because in my environment I still don't have installed mingw port for
some externals, build of following modules isn't tested:

Known issues:
The float, math and other related tests fail(under emulation and on nt5.1).
The reason is that C runtime function strtod() can't parse as example
numbers with exponent lower than -308, can't parse inf, nan, etc.. The
python source contain a file Python/strtod.c and my attempt to modify
and use it didn't succeed.
Also the configure lack test for "working" strtod and I guess that tests
fail on posix systems without C99 support.
The library mingwex from mingw runtime version 3.15 has working
replacement for stdtod function.
For now my patch will not address old strtod function.

In mingw definition of some functions(as example getaddrinfo and
getnameinfo) depend of value of WINVER. This issue is commented in the
patch. If user define WINVER geater the 0x0500 to CPPFLAGS at configure
time the build will use C-runtime function otherwise fake-function from
getaddrinfo.c and getnameinfo.c. As is commented in the patch if program
is linked with C-runtime functions I expect to fail to run on w2k.
I prefer to left selection of C-runtime for future mingw patches.

The other issue is that I see failure for some tests that use sockets.
This problem isn't investigated yet.

We may group some changes in configure but this require to reorder some
commands as example '--with-pydebug'. Since this isn't mingw issue I
prefer don't change current order. Another example is LIBM - it is set
by configure but setup.py use own rule.

components: Build
files: python-trunk.patch-MINGW
messages: 73239
nosy: rpetrov
severity: normal
status: open
title: cross building python for mingw32 with distutils
type: feature request
versions: Python 2.7
Added file: http://bugs.python.org/file11491/python-trunk.patch-MINGW

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list