[Patches] [ python-Patches-1006238 ] cross compile patch

SourceForge.net noreply at sourceforge.net
Tue Jul 4 18:48:45 CEST 2006


Patches item #1006238, was opened at 2004-08-09 17:05
Message generated for change (Comment added) made by goertzen
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1006238&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Daniel Goertzen (goertzen)
Assigned to: Nobody/Anonymous (nobody)
Summary: cross compile patch

Initial Comment:
Here's a cross compile patch I prepared a while ago but
never got around to submitting.  I've been using it
happily for months to cross compile python for embedded
systems.

Below is a descriptive excerpt from the patch.  Also
note that the patch modifies configure.in, but not
configure.  You will need to regenerate configure with
something like 

autoconf configure.in >configure

This patch is inpsired from work by Klaus Reimer at
http://www.ailis.de/~k/docs/crosscompiling/python.php

+ Cross Compiling
+ ---------------
+ 
+ Python can be cross compiled by supplying different
--host and --build
+ parameters to configure.  (Python is compiled on the
"build" system
+ and executed on the "host" system, in case you forgot
:).  Python is
+ tricky to cross compile because it needs to execute
parts of itself
+ during construction.  To work around this, make's
VPATH feature is
+ used to compile a native python in the subdirectory
"buildpython".
+ When parts of python need to be executed during
construction, the
+ "buildpython" versions are used.
+ 
+ A consequence of using the VPATH feature is that you
may not do a
+ cross compile build in the source directory.  In
other words, do this:
+ 
+ mkdir mydir
+ cd mydir
+ ../Python/configure --host=powerpc-405-linux-gnu
--build=i686-pc-linux-gnu
+ make
+ 
+ Cross compiling works well under linux, mileage may
vary for other
+ platforms.
+ 
+ A few reminders on using configure to cross compile:
+ - Cross compile tools must be in the PATH.
+ - Cross compile tools must be prefixed with the host type
+ (ie powerpc-405-linux-gnu-cc,
powerpc-405-linux-gnu-ranlib, ...)
+ - CC, CXX, AR, and RANLIB must be undefined when
running configure and
+ make.  Configure will detect them.
+ 
+ If you need a cross compiler, check out Dan Kegel's
crosstool:
+ http://www.kegel.com/crosstool
+ 
+ 


----------------------------------------------------------------------

>Comment By: Daniel Goertzen (goertzen)
Date: 2006-07-04 11:48

Message:
Logged In: YES 
user_id=843814

This stuff is becoming hazy for me, but I'll try to offer
some ideas:

- Does it help if you put your build directory outside of
the source directory?

- Why is the system looking for struct.so when you told it
to be static?  I suspect static support is broken for a lot
of modules.  Does it go better when you take the static
switch out?  Assuming you have a few other apps in your
system besides python (a shell, busybox maybe), I question
wether static would actually save any memory.  It perhaps is
a little more complex to get all the libraries in their
correct location, but I'll bet coaxing a python system to be
static is way harder.

- It is normal for some things to bomb out while compiling
because the python build scripts always check the build
system for library availability.  In many cases you don't
need those broken libraries.  It is also often possible to
hack up the install scripts to work properly for a cross
compiled system.  That said, simple things like struct
should be working properly without hackery.

Cheers,
Dan.

----------------------------------------------------------------------

Comment By: matt (mattcomms)
Date: 2006-07-04 04:08

Message:
Logged In: YES 
user_id=1543599

Hello, Wondering if your still offering some support.   
   
As people have reported success with 2.4.2 I decided to use  
that version. I have been carefull to follow the steps as  
described but still having some difficulty.  
  
The steps I take are:  
  
patch -p3 < ../python-patch  
autoconf configure.in >configure  
mkdir cross-build  
cd cross-build  
../configure --host=cris-axis-linux-gnu 
--build=i486-slackware-linux  
  
I also edit the LINKCC line in the makefile and include  
-static (as I want to run on devboard with limited memory  
and not to worry about shared libraries)  
  
make throws lots of errors like:  
  
*** WARNING: renaming "struct" since importing it failed:  
build/lib.linux-i686-2.4/struct.so: cannot open shared  
object file: No such file or directory  
building 'regex' extension  
cris-axis-linux-gnu-gcc -DNDEBUG -g -O3 -Wall  
-Wstrict-prototypes -fPIC -fno-strict-aliasing -I.  
-I/root/Python-2.4.2/./Include -I/usr/local/include  
-I/root/Python-2.4.2/cross-build/Include  
-I/root/Python-2.4.2/cross-build/buildpython  
-c /root/Python-2.4.2/Modules/regexmodule.c -o  
build/temp.linux-i686-2.4/regexmodule.o  
cris-axis-linux-gnu-gcc -DNDEBUG -g -O3 -Wall  
-Wstrict-prototypes -fPIC -fno-strict-aliasing -I.  
-I/root/Python-2.4.2/./Include -I/usr/local/include  
-I/root/Python-2.4.2/cross-build/Include  
-I/root/Python-2.4.2/cross-build/buildpython  
-c /root/Python-2.4.2/Modules/regexpr.c -o  
build/temp.linux-i686-2.4/regexpr.o  
cris-axis-linux-gnu-gcc -shared  
build/temp.linux-i686-2.4/regexmodule.o  
build/temp.linux-i686-2.4/regexpr.o -L/usr/local/lib -o  
build/lib.linux-i686-2.4/regex.so  
  
and  
  
cris-axis-linux-gnu-gcc -DNDEBUG -g -O3 -Wall  
-Wstrict-prototypes -fPIC -fno-strict-aliasing -I.  
-I/root/Python-2.4.2/./Include -I/usr/local/include  
-I/root/Python-2.4.2/cross-build/Include  
-I/root/Python-2.4.2/cross-build/buildpython  
-c /root/Python-2.4.2/Modules/_ssl.c -o  
build/temp.linux-i686-2.4/_ssl.o  
/root/Python-2.4.2/Modules/_ssl.c:30:25: openssl/rsa.h: No  
such file or directory  
  
make install prefix=/root/Python-2.4.2/cross-build  
  
throws the same errors but finishes.  
  
I then mount cross-build via nfs and run python  
  
Python 2.4.2 (#2, Jun 28 2006, 18:35:28)  
[GCC 3.2.1 Axis release R61/1.61] on linux2  
Type "help", "copyright", "credits" or "license" for more  
information.  
>>> import time  
import time  
Traceback (most recent call last):  
  File "<stdin>", line 1, in ?  
ImportError: /mnt/flash/Python/lib/python2.4/lib-dynload/time.so:  
undefined symbol: PyExc_IOError  
>>>  
  
>>> import socket  
import socket  
Traceback (most recent call last):  
  File "<stdin>", line 1, in ?  
  File "/mnt/flash/Python/lib/python2.4/socket.py", line  
45, in ?  
    import _socket  
ImportError: /mnt/flash/Python/lib/python2.4/lib-dynload/_socket.so:  
undefined symbol: PyObject_GenericGetAttr  
  
Any hint as to what might be causing these  errors would be  
most appreciated.  
  
Cheers  
  
Matthew  

----------------------------------------------------------------------

Comment By: Daniel Goertzen (goertzen)
Date: 2006-03-07 15:45

Message:
Logged In: YES 
user_id=843814

I use this command:

make install prefix=/my/temp/dir

----------------------------------------------------------------------

Comment By: Nicolas (nico0438)
Date: 2006-03-07 15:30

Message:
Logged In: YES 
user_id=1330285

Hi again,

It seems I'm successfull in cross-compiling python.
But I'm not sure of what I have to save on the host platform.
There is python executable of course. But there is no lib
directory with .so files and .py files in the directory
where python is built.
Can you tell me what files/directories you save on your host
platform ?
Thanks again for your support.
Nicolas

----------------------------------------------------------------------

Comment By: Daniel Goertzen (goertzen)
Date: 2006-03-06 09:12

Message:
Logged In: YES 
user_id=843814

Hi hugo.

The problems you are having are less python related and more
system-building/cross-development issues, and this is the
wrong forum for hashing out this stuff out.

I recommend building a linux-from-scratch system
(http://www.linuxfromscratch.org/) on an ordinary PC to
learn how the insides of linux machine works and how to fix
things when they go wrong.

Good luck,
Dan.

----------------------------------------------------------------------

Comment By: hugo (hugo_koopmans)
Date: 2006-03-04 14:02

Message:
Logged In: YES 
user_id=1229536

Hi there,
I am trying to cross compile python on the fox board
(etrax). now everything seemed to go well but when running
the python binary is says:
./python: error while loading shared libraries:
libstdc++.so.5: cannot load shared object file: No such file
or directory

I only ftp-ed the python bin not the (much bigger)
libpython2.4a ....
seems this is a library archive of some kind ?
do i need it?
how do i use it then?

thanx a million if you can help me out.

hugo

----------------------------------------------------------------------

Comment By: Daniel Goertzen (goertzen)
Date: 2006-02-24 16:32

Message:
Logged In: YES 
user_id=843814

You didn't apply the patch properly.  See the other messages
about the use of the -p option.  Start with a fresh source
tree and use:

cd Python-2.4.2
patch -p3 <../python-patch.txt

----------------------------------------------------------------------

Comment By: Nicolas (nico0438)
Date: 2006-02-24 15:37

Message:
Logged In: YES 
user_id=1330285

If I try to aplly the patch again, I get messages saying a
patch has allready been applied.
To apply the patch I executed the following :

cd Python-2.4.2
patch <../python-patch.txt



----------------------------------------------------------------------

Comment By: Daniel Goertzen (goertzen)
Date: 2006-02-24 15:30

Message:
Logged In: YES 
user_id=843814

The build system python should be in "MyPython/buildpython",
and the host python should just be in "MyPython"

Are you sure you applied the patch right?  What you're
getting is what would happen if you didn't apply the patch
at all.

----------------------------------------------------------------------

Comment By: Nicolas (nico0438)
Date: 2006-02-24 15:19

Message:
Logged In: YES 
user_id=1330285

Yes I guess.

But it seems that all I get is for the build system.
If I try to execute on the build system ./python it runs.
Also, there is no hostbuild subdirectory. Is it ok ?

Nicolas

----------------------------------------------------------------------

Comment By: Daniel Goertzen (goertzen)
Date: 2006-02-24 15:14

Message:
Logged In: YES 
user_id=843814

> I then cd to a "MyPython" directory.
> I call ../Python-2.4.2/configure ...

Then your files should be in your "MyPython" directory.

----------------------------------------------------------------------

Comment By: Nicolas (nico0438)
Date: 2006-02-24 14:54

Message:
Logged In: YES 
user_id=1330285

Me again, sorry...


What Should I get after compilation has finished ?
Where are the files for my target ?

Nicolas

----------------------------------------------------------------------

Comment By: Nicolas (nico0438)
Date: 2006-02-24 13:55

Message:
Logged In: YES 
user_id=1330285

Hi again,

I finaly dad enought time to work on my project.
So, I compiled autoconf tools for my system.
Then, I rerun all the steps successfully.

Thanks again for your help.

Nicolas

----------------------------------------------------------------------

Comment By: nicolas pinault (nico38)
Date: 2006-02-24 09:12

Message:
Logged In: YES 
user_id=1081828

You are right, I have misunderstood --build and --host
signification.

And I also forgotten to run autoconf partialy because I
don't have autoconf installed on my system. So I didn't pay
attention it didn't run.

Thanks for your help.
I'll let you know if this works in a few days.

Nicolas

----------------------------------------------------------------------

Comment By: Daniel Goertzen (goertzen)
Date: 2006-02-24 08:51

Message:
Logged In: YES 
user_id=843814

Did you run autoconf after patching as instructed?

Also, your --build and --host options seem backwards.  host
is the arch you want python to run on, build is the arch you
build python on.

Cheers,
Dan.

----------------------------------------------------------------------

Comment By: nicolas pinault (nico38)
Date: 2006-02-24 06:38

Message:
Logged In: YES 
user_id=1081828

Hi,

I try to cross compile Python-2.4.2 for an etrax processor.
I have successfully applied the patch to Python sources.
I then cd to a "MyPython" directory.
I call ../Python-2.4.2/configure --build cris --host i386-linux
It works ok (did not see an error)
I all make
I get the following error after a while :
make : *** No rule to make target '@BUILDPGEN@', needed by
'../Python-2.4.2/Include/graminit.h'. Stop.

Any Idea ?
Nicolas

----------------------------------------------------------------------

Comment By: xudong (xudong888)
Date: 2006-01-10 08:45

Message:
Logged In: YES 
user_id=1420135

I have resolved my question,now I can run python on the 
mips machine,and can run some binary build by python.Can I 
also cross compile pygame for python?But the pygame source 
does't has 'configure' and 'Makefiel' files,setup.py 
instead in source package.How should I do?
Thanks :-)

----------------------------------------------------------------------

Comment By: xudong (xudong888)
Date: 2006-01-06 05:08

Message:
Logged In: YES 
user_id=1420135

If you have accounts of MSN,please add me.My accounts is 
xudong_888 at hotmail.com. I am very impatient.I hope I can 
get some help from you.Thanks.

----------------------------------------------------------------------

Comment By: xudong (xudong888)
Date: 2006-01-06 04:57

Message:
Logged In: YES 
user_id=1420135

Thank you for your rapid answer.But I still can't solve my 
question.I can't find the whole instructions. Can you give 
me the whole instructions by EMAIL,my email is 
xudong888 at 163.com,thanks.
My "host" system is mips embbed_linux,the release of the 
linux kernel is 2.4.xx and the CPU type is MIPS 4Kc.There 
are no python installed on this system.My "build" system is 
i86 and operate system is RedHat9.0,and has installed 
python2.4.1 and cross-compiling Tools mipsel-linux-gcc.I 
can cross compile C program for the "host" system.I have 
writed some program with python language.I can get the 
binary from python script by use freeze.py on the "build" 
system. Now I want to run the binary on the "host" 
system.Can you tell me how should I do.
in addition,I don't know what parameter I should input to 
use this patch.
finally,I'm sorry for my poor English.I like python very 
much,but I can't get help in chinese.

----------------------------------------------------------------------

Comment By: Daniel Goertzen (goertzen)
Date: 2006-01-05 11:24

Message:
Logged In: YES 
user_id=843814

After configure you run "make".

But did you use configure as the instructions say?  You
cannot just use "./configure".

Good luck,
Dan.

----------------------------------------------------------------------

Comment By: xudong (xudong888)
Date: 2006-01-05 10:55

Message:
Logged In: YES 
user_id=1420135

I am a Chinese and my English is very poor.I'm sorry if 
what I said is wrong.My question is What should I do 
after './configure'.Before this I have done patch and 
autoconf.

Thanks :-)

----------------------------------------------------------------------

Comment By: David Lambert (jdalambert)
Date: 2005-11-08 10:06

Message:
Logged In: YES 
user_id=845425

Oops! All works fine now. Thanks :-)

----------------------------------------------------------------------

Comment By: Daniel Goertzen (goertzen)
Date: 2005-11-08 09:09

Message:
Logged In: YES 
user_id=843814

patch isn't lying about wrong -p options.  Use -p3 instead
of -p0.

Cheers,
Dan.

----------------------------------------------------------------------

Comment By: David Lambert (jdalambert)
Date: 2005-11-08 08:26

Message:
Logged In: YES 
user_id=845425

Thanks for the quick reply, and sorry for the confusion.

I DID try the cross compile in a sub directory. That failed
with the same error. I then tried a non-cross build in the
main directory, that also failed (which was my previous
post). Here is my complete transcript after untarring Python:

[dlambert at lambsys Python-2.4.2]$ patch -p0 < ../python-patch
can't find file to patch at input line 3
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|*** python-cvs-pristine/dist/src/README        Fri Mar  5
08:33:21 2004
|--- python/dist/src/README     Mon Apr  5 14:30:23 2004
--------------------------
File to patch: README
patching file README
Hunk #1 succeeded at 1130 (offset 30 lines).
can't find file to patch at input line 48
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|*** python-cvs-pristine/dist/src/configure.in  Sun Mar 21
17:45:41 2004
|--- python/dist/src/configure.in       Mon Apr  5 16:15:07 2004
--------------------------
File to patch: configure.in
patching file configure.in
Hunk #2 succeeded at 609 (offset 58 lines).
Hunk #3 succeeded at 3072 (offset 112 lines).
can't find file to patch at input line 113
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|*** python-cvs-pristine/dist/src/Makefile.pre.in       Thu
Mar 18 01:51:27 2004
|--- python/dist/src/Makefile.pre.in    Mon Apr  5 15:56:00 2004
--------------------------
File to patch: Makefile.pre.in
patching file Makefile.pre.in
Hunk #2 succeeded at 163 (offset 3 lines).
Hunk #4 succeeded at 309 (offset 5 lines).
Hunk #6 succeeded at 470 (offset 5 lines).
Hunk #7 succeeded at 624 (offset 1 line).
Hunk #8 succeeded at 839 (offset 7 lines).
Hunk #9 succeeded at 923 (offset 1 line).
Hunk #10 succeeded at 969 (offset 7 lines).
can't find file to patch at input line 309
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|*** python-cvs-pristine/dist/src/setup.py      Sun Mar 21
12:59:46 2004
|--- python/dist/src/setup.py   Mon Apr  5 15:20:55 2004
--------------------------
File to patch: setup.py
patching file setup.py
Hunk #1 succeeded at 198 (offset -2 lines).
patching file python/dist/src/config.guess
patching file python/dist/src/config.sub
[dlambert at lambsys Python-2.4.2]$
[dlambert at lambsys Python-2.4.2]$
[dlambert at lambsys Python-2.4.2]$ autoconf configure.in >
configure
[dlambert at lambsys Python-2.4.2]$ mkdir cross-build
[dlambert at lambsys Python-2.4.2]$ cd cross-build
[dlambert at lambsys cross-build]$ ../configure
--host=arm-linux --build=i686-pc-li nux-gnu
configure: error: cannot run /bin/sh ../config.sub
[dlambert at lambsys cross-build]$



----------------------------------------------------------------------

Comment By: Daniel Goertzen (goertzen)
Date: 2005-11-08 08:05

Message:
Logged In: YES 
user_id=843814

You can't configure in the source directory with the cross
compile patch.  This is explained in the directions.

----------------------------------------------------------------------

Comment By: David Lambert (jdalambert)
Date: 2005-11-07 20:12

Message:
Logged In: YES 
user_id=845425

Hmm. not so much luck here. I get the following error after
patching then autoconf. I am running on Fedora Core 4.

Any suggestions?

[dlambert at lambsys Python-2.4.2]$ autoconf configure.in
>configure
[dlambert at lambsys Python-2.4.2]$ ./configure
configure: error: cannot run /bin/sh ./config.sub
[dlambert at lambsys Python-2.4.2]$

----------------------------------------------------------------------

Comment By: Robsa (robsa)
Date: 2005-10-13 06:54

Message:
Logged In: YES 
user_id=1277505

Just thought I'd let you know that this patch works against
Python 2.4.2. I had Python running on my custom AT91RM9200
board (ARM920T core) in about 20 minutes.

*snaps* for Daniel!!

----------------------------------------------------------------------

Comment By: Daniel Goertzen (goertzen)
Date: 2005-04-13 12:36

Message:
Logged In: YES 
user_id=843814

It still works for me, so I've had limited interest in
working on it further.

I think a "real" cross compile patch will involve
significant refactoring of distutils and the main setup.py
script.  Should this start with a PEP?


----------------------------------------------------------------------

Comment By: Ned Ludd (solarx)
Date: 2005-04-09 15:43

Message:
Logged In: YES 
user_id=148412

Any progress with this on? 
python and perl are two of the last major things to overcome
in the x-compile world.

----------------------------------------------------------------------

Comment By: Mike Frysinger (vapier)
Date: 2004-10-26 11:00

Message:
Logged In: YES 
user_id=114429

we've been using this with uClibc for a while now and it
works great ...

ive personally built (on amd64) & deployed (on
x86/ppc/arm/mips/sh4) python ... would great if this finally
made it into the official sources :)

----------------------------------------------------------------------

Comment By: Jeff Epler (jepler)
Date: 2004-10-20 20:08

Message:
Logged In: YES 
user_id=2772

This patch applies cleanly to today's CVS head.

If building in the source directory while cross compiling
fails, please make configure complain about it.

This patch makes the build process invoke make recursively,
which is a big minus.  I'd rather see pgen built with a
HOSTCC and just assume python is available on $PATH for the
setup.py step.  There's no reason to build all the shared
modules in buildpython, either, which would speed things up
a fair bit.

On to how it worked for me:
Not cross-compiling, this produces no unexpected test
failures in "make test". (redhat9)

Cross compiling for i386-pc-bsdi2.1 everything goes fine
until it tries to run buildpython and make shared modules,
but shared modules don't work in the first place on bsdi2. 
I did not test the resulting Python binary.

Cross compiling for i386-redhat-linux (libc6) some
extensions fail to build, but this could be because my
header files for the cross-development environment are not
complete.  Running "make test" tries to invoke the
"buildpython/python" and doesn't work.  Running it manually
I get some skips due to modules that did not build, but
everything that did build seems to pass. (OK, so I wrote
this before the tests completed, but they're off to a good
start)

I currently cross-compile python 2.3 for win32 (mingw), and
until recently cross-compiled it for bsdi2 and redhat6. 
However, I did not use configure or the included makefiles
to do this (in order to integrate in a non-recursive build
procedure that builds several packages), so this patch is
unlikely to benefit me directly.

I don't think this patch is ready to be applied.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1006238&group_id=5470


More information about the Patches mailing list