Problem building Python 2.7.5 with separate sysroot
Hi all. I'm trying to build Python 2.7.5 on a GNU/Linux (Linux Mint 14) system, but using a different sysroot (that is, a separate <d>/usr/include, <d>/usr/lib, etc., not the real one for my system). I have shell script wrappers around GCC and its various tools that invoke it with the right paths to force this to happen, and when I call Python's configure I send along "CC=sysroot-gcc", etc. for all the various tools. Note that it's not really a cross-compilation because the target is also a GNU/Linux system on the same hardware architecture. The majority of Python builds just fine like this. However, I'm having serious problems building modules such as fcntl, etc. Looking at the output from the makefile, I can see that somehow, someone is forcibly adding "-I/usr/include/x86_64-linux-gnu" to the link line: building 'termios' extension sysroot-gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I/usr/include/x86_64-linux-gnu -I/common/sysroot/tools/usr/include -I/common/sysroot/tools/usr/include-fixed -I/usr/local/include -I/home/workspaces/psmith/python/obj/src/Python-2.7.5/Include -I/home/workspaces/psmith/python/obj/bld/python -c /home/workspaces/psmith/python/obj/src/Python-2.7.5/Modules/termios.c -o build/temp.linux-x86_64-2.7/home/workspaces/psmith/python/obj/src/Python-2.7.5/Modules/termios.o This fails miserably because the headers in /usr/include/x86_64-linux-gnu do not play at all nicely with my other sysroot headers. Ditto for other extensions like fcntl, etc. I've searched high and low in the Python source, generated makefiles, config.log, etc. and I cannot find where this -I flag is coming from anywhere. I found the --oldincludedir flag to configure and set it to point into my sysroot as well, but that didn't help: the /usr/include still appears when building these extensions. Can anyone tell me where Python is getting these -I flags and what I need to do to tell it to NOT use those flags when building extensions? I'd also like to remove the -I/usr/local/include, although this is not actually causing me problems right now.
In article <1369986770.4119.43.camel@homebase>, Paul Smith <paul@mad-scientist.net> wrote:
Hi all. I'm trying to build Python 2.7.5 on a GNU/Linux (Linux Mint 14) system, but using a different sysroot (that is, a separate <d>/usr/include, <d>/usr/lib, etc., not the real one for my system).
This list is for the development of Python itself, not about using or installing it. Python-list (AKA comp.lang.python) is the right list to ask such questions. That said ...
However, I'm having serious problems building modules such as fcntl, etc. Looking at the output from the makefile, I can see that somehow, someone is forcibly adding "-I/usr/include/x86_64-linux-gnu" to the link line: [...]
... include file and library file selections for building standard library modules are handled by the top-level setup.py file in the source tree. That's where /usr/local/... is added and chances are that the above header is being added by add_gcc_paths() in setup.py. -- Ned Deily, nad@acm.org
On Fri, 2013-05-31 at 01:21 -0700, Ned Deily wrote:
In article <1369986770.4119.43.camel@homebase>, Paul Smith <paul@mad-scientist.net> wrote:
Hi all. I'm trying to build Python 2.7.5 on a GNU/Linux (Linux Mint 14) system, but using a different sysroot (that is, a separate <d>/usr/include, <d>/usr/lib, etc., not the real one for my system).
This list is for the development of Python itself, not about using or installing it. Python-list (AKA comp.lang.python) is the right list to ask such questions. That said ...
However, I'm having serious problems building modules such as fcntl, etc. Looking at the output from the makefile, I can see that somehow, someone is forcibly adding "-I/usr/include/x86_64-linux-gnu" to the link line: [...]
... include file and library file selections for building standard library modules are handled by the top-level setup.py file in the source tree. That's where /usr/local/... is added and chances are that the above header is being added by add_gcc_paths() in setup.py.
Yes, thank you. It seems to me (keeping with the theme of this mailing list) that the add_multiarch_paths() function in setup.py is not right. The first step, which asks the compiler about multi-arch, is OK because it's using my alternate compiler which reports no multiarch. But then it proceeds to run the local host version of dpkg-architecture. I see that it adds the -t flag if cross-compiling, which I'm not, but even that is not fixing the issue. If you're building on a system which is Debian derived with multi-arch support you will ALWAYS have your local "/usr/include" (plus some multiarch suffix) -- and '/usr/lib' + multiarch -- added to your include and lib paths; this is not good. My case may be unusual but even in a more formal cross-compilation environment it's not good to add /usr/include/..., or base such a decision on the behavior of the _build_ system.
In article <1370000129.4119.53.camel@homebase>, Paul Smith <paul@mad-scientist.net> wrote:
It seems to me (keeping with the theme of this mailing list) that the add_multiarch_paths() function in setup.py is not right. [...]
If you think there is a problem, please open an issue for it on the Python bug tracker: http://bugs.python.org. Thanks! -- Ned Deily, nad@acm.org
Quoting Paul Smith <paul@mad-scientist.net>:
My case may be unusual but even in a more formal cross-compilation environment it's not good to add /usr/include/..., or base such a decision on the behavior of the _build_ system.
The offical procedure to cover "unusual" cases is to edit Modules/Setup. If you are not happy with the way in which modules are build, you can override all flags on a per-module basis there.
It seems to me (keeping with the theme of this mailing list) that the add_multiarch_paths() function in setup.py is not right.
Well, ..., yes. For the last two decades, the build process of Python was *always* wrong, and it always ever will be, in the sense that it doesn't support all cases that people come up with. The only way to deal with this, unfortunately, is to patch the build process for each new use case discovered. As Ned says: if you come up with a patch, don't hesitate to post it to the tracker. Regards, Martin
participants (3)
-
martin＠v.loewis.de -
Ned Deily -
Paul Smith