[Python-Dev] Cross compiling Python (for Android)
Guido van Rossum
guido at python.org
Sat Oct 25 05:15:42 CEST 2014
Hi Frank,
Nobody has responded to you yet, but don't feel discouraged by that! The
core Python development group consists mostly of people who don't develop
mobile apps in their day jobs, and they may feel reluctant to maintain
changes that they can't personally test. (And I don't expect it would be
easy to set up Android or iOS buildbots either.)
But there are definitely people trying to use Python to develop mobile apps
(e.g. Kivy). Today there was a similar post to python-ideas about this. I
think the world may soon be eager to develop mobile apps in Python (the
platforms are maturing and the processors are getting faster), and Python
should be ready for this change in attitude.
Hopefully we can get some good patches in for the next bugfix releases of
Python 2.7 and 3.4, as well as the upcoming 3.5 alphas and betas.
Of course, the Kivy approach might work for some time yet (they have a set
of patches for Python 2.7.1 or 2.7.2), but it would be better if that
wasn't necessary, and Python could be build (with the right dev
environment) for iOS and Android.
A word of advice: the specific patches you have should probably be
submitted to the Python issue tracker (bugs.python.org). Also, several
smaller patches are more likely to be reviewed and checked in timely than
one mega-patch.
Good luck!
--Guido
On Thu, Oct 23, 2014 at 12:22 PM, Frank, Matthew I <
matthew.i.frank at intel.com> wrote:
> This email is about my experience getting CPython (3.4.1) to
>
> cross-compile and run on x86 Android (4.4.2 with sdk 19 and ndk-r9).
>
> I know that Android is not a supported architecture (and I won't
>
> regale you with stories about the complete locale and mbstowcs support
>
> I had to borrow from FreeBSD to get it working). The purpose of this
>
> email is that several things I found are arguably "bugs" in the Python
>
> build system or code when it comes to cross-compiling that are exposed
>
> by Android's poor Posix support. I'd like some advice about what kind
>
> of patch (if any) would be most suitable for fixing the problems on
>
> the Python side.
>
>
>
> Just to be complete: I'm configuring with
>
>
>
> CPPFLAGS=-I../my-locale ../Python-3.4.1/configure --enable-shared
>
> --prefix=/path/to/install/dir --build=x86_64-linux-gnu
>
> --host=i686-linux-android --disable-ipv6 ac_cv_file__dev_ptmx=no
>
> ac_cv_file__dev_ptc=no ac_cv_little_endian_double=yes
>
>
>
> (The CPPFLAGS addition is to get the headers for my locale fixes
>
> instead of the default Android ones. ac_cv_file__dev_ptmx=no and
>
> ac_cv_file__dev_ptc=no are because I don't have /dev/whatever on my
>
> build machine. ac_cv_little_endian_double is because configure for
>
> cross builds can't figure out the endianness of doubles on the host
>
> (because it is running on the build machine not the host.) (For ARM
>
> it would be ac_cv_mixed_endian_double=yes.)
>
>
>
> I've gotten to the point where `make; make install` succeeds up to the
>
> point of building something that runs on my Android system (from the
>
> command line) and `python -m test` runs 388 tests, with 321 ok, 24
>
> test failures and 43 tests skipped (the skips mostly due, I think, to
>
> me not yet having installed the right cross-building support for
>
> things like bz2 and dbm.)
>
>
>
> 1. `make` succeeds but `make install` always fails at the end with
>
> something having to do with being unable to run "ensurepip"
>
> (presumably because ensurepip requires modules that only run on the
>
> host, not the build module.) So it seems this should be wrapped in
>
> a test for cross compilation, but I haven't looked at exactly what
>
> yet. The error is:
>
>
>
> /linux-python/bin/python3.4: Error while finding spec for
>
> 'ensurepip.__main__' (<class 'ImportError'>:
>
> /build-directory/build/lib.linux-i686-3.4/time.cpython-34m.so:
>
> wrong ELF class: ELFCLASS32); 'ensurepip' is a package and cannot
>
> be directly executed make: *** [install] Error 1
>
>
>
> 2. setup.py is missing -lm flag for several modules. On Linux this
>
> problem is hidden because libm is already loaded by the executable
>
> calling dlopen(), but Android's loader deals with unknown symbols
>
> differently (searches only the libs explicitly linked against the
>
> module being loaded.) http://bugs.python.org/issue21668 reports
>
> the problem for selectmodule (can't find ceil()) and timemodule
>
> (fmod() and floor()). But there are at least two more: audioop
>
> fails to load because it needs floor() and ctypes_test fails to
>
> load because it needs sqrt(). I'll happily update the patch in
>
> 21668.
>
>
>
> Is there any fundamental objection to adding the -lm flag to the
>
> link step where it is necessary?
>
>
>
> 3. What is ossaudiodev? It tries to include "sys/soundcard.h", which
>
> I don't have on my system. (The rule in setup.py is
>
> wrapped in a test for host of Linux/FreeBSD/Darwin, but Android x86
>
> gets configured with --host=i686-linux-android so to turn it off
>
> requires an extra test for "and not cross_compiling".)
>
>
>
> Can I just turn off ossaudiodev for cross compiling or might
>
> someone want it in a different type of cross build? (In which case
>
> I think I'll have to write some kind autoconf rule for it, which I
>
> don't quite know how to do yet.)
>
>
>
> 4. Module _decimal is failing to compile. The problem is that it has
>
> a header called memory.h. Android's libc has the problem that
>
> /usr/include/stdlib.h includes <memory.h>. But the build system
>
> puts -I. on the include path before the system dirs (as it should)
>
> so when compiling _decimal, Modules/_decimal/libmpdec/memory.h gets
>
> found instead of /usr/include/memory.h. Shiz has a patch here:
>
>
> https://github.com/rave-engine/python3-android/blob/master/mk/python/3.3.5/p\
>
> ython-3.3.5-android-libmpdec.patch
>
> (which renames memory.h -> mpmemory.h) but I don't know
>
>
>
> a. Is there a tracker for this yet? and
>
> b. Is Shiz's fix the desired one or should I be looking for
>
> another approach? (Maybe modifying the -I flags for the build
>
> of just the build of _decimal or something?)
>
>
>
> 5. I'm not sure what test configure is actually doing for gethostby*()
>
> in a cross-compile environment. In any case Android has a bug
>
> where gethostbyaddr_r() is declared in the headers, but not
>
> actually implemented in libc. So I have to modify my pyconfig.h by
>
> hand to define HAVE_GETHOSTBYNAME and undef HAVE_GETHOSTBYNAME_R
>
> and HAVE_GETHOSTBYNAME_R_6_ARG.
>
>
>
> Is there a variable (like ac_cv_little_endian_double) that I can
>
> give to `configure` to make it set HAVE_GETHOSTBYNAME* the way I
>
> need? If so I've been unable to figure it out.
>
>
>
> 6. Android's <pwd.h> header mysteriously leaves the pw_gecos field out
>
> of struct passwd. Is a fix like defining a new variable
>
> HAVE_BROKEN_GECOS_FIELD the appropriate way to go with this? (If
>
> this is an okay solution then the patch to Modules/pwdmodule.c is
>
> shown below, but I still have to figure out how to patch
>
> configure.ac to test for the condition and set the variable
>
> appropriately, so a pointer to a similar block of code in
>
> configure.ac would be appreciated.)
>
>
>
> Sorry for the TL;DR. I appreciate your having taken the time to read
>
> this far.
>
>
>
> Thanks,
>
> -Matt
>
>
>
> Proposed patch for pwdmodule.c:
>
>
>
> --- a/Modules/pwdmodule.c 2014-05-19 00:19:39.000000000 -0500
>
> +++ b/Modules/pwdmodule.c 2014-10-21 18:00:35.676331205 -0500
>
> @@ -57,6 +57,10 @@
>
> }
>
> }
>
>
>
> +#if defined(HAVE_BROKEN_GECOS_FIELD)
>
> +static char fakePwGecos[256] = "";
>
> +#endif
>
> +
>
> static PyObject *
>
> mkpwent(struct passwd *p)
>
> {
>
> @@ -72,7 +76,11 @@
>
> SETS(setIndex++, p->pw_passwd);
>
> PyStructSequence_SET_ITEM(v, setIndex++, _PyLong_FromUid(p->pw_uid));
>
> PyStructSequence_SET_ITEM(v, setIndex++, _PyLong_FromGid(p->pw_gid));
>
> +#if !defined(HAVE_BROKEN_GECOS_FIELD)
>
> SETS(setIndex++, p->pw_gecos);
>
> +#else
>
> + SETS(setIndex++, fakePwGecos);
>
> +#endif
>
> SETS(setIndex++, p->pw_dir);
>
> SETS(setIndex++, p->pw_shell);
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
--
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141024/292a4aad/attachment-0001.html>
More information about the Python-Dev
mailing list