[Python-Dev] Support of the Android platform

Tue Jan 23 12:17:05 EST 2018

Ok, I created https://bugs.python.org/issue32637
"Android: set sys.platform and os.name to android"

Victor

2018-01-23 18:01 GMT+01:00 Brett Cannon <brett at python.org>:
>
>
> On Mon, 22 Jan 2018 at 09:29 Victor Stinner <victor.stinner at gmail.com>
> wrote:
>>
>> Hi,
>>
>> I'm still talking with Paul Peny (pmpp on IRC) who is trying to build
>> the master branch of Python on Android, using cross-compilation or
>> directly on an Android device. I started to took notes since Android
>> is a complex platforms and it's not easy for me to remember
>> everything.
>>
>> http://vstinner.readthedocs.io/python_android.html
>>
>> Paul would like to support Android 4.4 Kitkat (API 19) just because
>> it's possible to find cheap devices running Android, but usually only
>> with old Android versions. Technically, it doesn't see difficult to
>> support API 19+ (instead of 21+), a few tiny patches are needed. But I
>> don't want to have a "full support" of API 19+, only basic support
>> like "make sure that the compilation doesn't fail", not "all tests
>> must pass".
>>
>> It seems like sys.platform == 'android' would be more appropriate
>> since Android is not Linux: different libc, different filesystems,
>> etc.
>
>
> I've had a similar thought myself.
>
> -Brett
>
>>
>>
>> While Xavier promotes cross-compilation, Paul would like to build
>> Python directly on Android to get pip and wheels.
>>
>> Honestly, I have no strong opinion, since I don't know well Android.
>> I'm trying to help everybody working on the Android support. IMHO it's
>> fine to support multiple ways to build Python for Android. It's not
>> like there is very obvious option which has no drawback... Cross
>> compilation is complex, getting a C compiler on Android also seems to
>> be complex. From my point of view, compared to a common Fedora Linux,
>> Android doesn't seem easy to use to develop on CPython...
>>
>> Victor
>>
>> 2017-12-10 15:19 GMT+01:00 Xavier de Gaye <xdegaye at gmail.com>:
>> > The following note is a proposal to add the support of the Android
>> > platform.
>> >
>> > The note is easier to read with clickable links at
>> > https://github.com/xdegaye/cagibi/blob/master/doc/android_support.rst
>> >
>> > Motivations
>> > ===========
>> >
>> > * Android is ubiquitous.
>> > * This would be the first platform supported by Python that is
>> > cross-compiled,
>> >   thanks to many contributors.
>> > * Although the Android operating system is linux, it is different from
>> > most
>> >   linux platforms, for example it does not use GNU libc and runs SELinux
>> > in
>> >   enforcing mode. Therefore supporting this platform would make Python
>> > more
>> >   robust and also would allow testing it on arm 64-bit processors.
>> > * Python running on Android is also a handheld calculator, a successor
>> > of
>> > the
>> >   slide rule and the `HP 41`_.
>> >
>> > Current status
>> > ==============
>> >
>> > * The Python test suite succeeds when run on Android emulators using
>> > buildbot
>> >   strenuous settings with the following architectures on API 24: x86,
>> > x86_64,
>> >   armv7 and arm64.
>> > * The `Android build system`_ is described in another section.
>> > * The `buildmaster-config PR 26`_ proposes to update ``master.cfg`` to
>> > enable
>> >   buildbots to run a given Android API and architecture on the
>> > emulators.
>> > * The Android emulator is actually ``qemu``, so the test suites for x86
>> > and
>> >   x86_64 last about the same time as the test suite run natively when
>> > the
>> >   processor of the build system is of the x86 family. The test suites
>> > for
>> > the
>> >   arm architectures last much longer: about 8 hours for arm64 and 10
>> > hours
>> > for
>> >   armv7 on a four years old laptop.
>> > * The changes that have been made to achieve this status are listed in
>> >   `bpo-26865`_, the Android meta-issue.
>> > * Given the cpu resources required to run the test suite on the arm
>> > emulators,
>> >   it may be difficult to find a contributed buildbot worker. So it
>> > remains
>> > to
>> >   find the hardware to run these buildbots.
>> >
>> > Proposal
>> > ========
>> >
>> > Support the Android platform on API 24 [1]_ for the x86_64, armv7 and
>> > arm64
>> > architectures built with NDK 14b.
>> >
>> > *API 24*
>> >   * API 21 is the first version to provide usable support for wide
>> > characters
>> >     and where SELinux is run in enforcing mode.
>> >
>> >   * API 22 introduces an annoying bug on the linker that prints
>> > something
>> > like
>> >     this when python is started::
>> >
>> >       ``WARNING: linker: libpython3.6m.so.1.0: unused DT entry: type
>> > 0x6ffffffe arg 0x14554``.
>> >
>> >     The `termux`_ Android terminal emulator describes this problem at
>> > the
>> > end
>> >     of its `termux-packages`_ gitlab page and has implemented a
>> >     ``termux-elf-cleaner`` tool to strip the useless entries from the
>> > ELF
>> >     header of executables.
>> >
>> >   * API 24 is the first version where the `adb`_ shell is run on the
>> > emulator
>> >     as a ``shell`` user instead of the ``root`` user previously, and the
>> > first
>> >     version that supports arm64.
>> >
>> > *x86_64*
>> >   It seems that no handheld device exists using that architecture. It is
>> >   supported because the x86_64 Android emulator runs fast and therefore
>> > is a
>> >   good candidate as a buildbot worker.
>> >
>> > *NDK 14b*
>> >   This release of the NDK is the first one to use `Unified headers`_
>> > fixing
>> >   numerous problems that had been fixed by updating the Python configure
>> > script
>> >   until now (those changes have been reverted by now).
>> >
>> > Android idiosyncrasies
>> > ======================
>> >
>> > * The default shell is ``/system/bin/sh``.
>> > * The file system layout is not a traditional unix layout, there is no
>> >   ``/tmp`` for example. Most directories have user restricted access,
>> >   ``/sdcard`` is mounted as ``noexec`` for example.
>> > * The (java) applications are allocated a unix user id and a
>> > subdirectory on
>> >   ``/data/data``.
>> > * SELinux is run in enforcing mode.
>> > * Shared memory and semaphores are not supported.
>> > * The default encoding is UTF-8.
>> >
>> > Android build system
>> > ====================
>> >
>> > The Android build system is implemented at `bpo-30386`_ with `PR 1629`_
>> > and
>> > is documented by its `README`_. It provides the following features:
>> >
>> > * To build a distribution for a device or an emulator with a given API
>> > level
>> >   and a given architecture.
>> > * To start the emulator and
>> >     + install the distribution
>> >     + start a remote interactive shell
>> >     + or run remotely a python command
>> >     + or run remotely the buildbottest
>> > * Run gdb on the python process that is running on the emulator with
>> > python
>> >   pretty-printing.
>> >
>> > The build system adds the ``Android/`` directory and the
>> > ``configure-android``
>> > script to the root of the Python source directory on the master branch
>> > without
>> > modifying any other file. The build system can be installed, upgraded
>> > (i.e.
>> > the
>> > SDK and NDK) and run remotely, through ssh for example.
>> >
>> > The following external libraries, when they are configured in the build
>> > system,
>> > are downloaded from the internet and cross-compiled (only once, on the
>> > first
>> > run of the build system) before the cross-compilation of the extension
>> > modules:
>> >
>> > * ``ncurses``
>> > * ``readline``
>> > * ``sqlite``
>> > * ``libffi``
>> > * ``openssl``, the cross-compilation of openssl fails on x86_64 and
>> > arm64
>> > and
>> >   this step is skipped on those architectures.
>> >
>> > The following extension modules are disabled by adding them to the
>> > ``*disabled*`` section of ``Modules/Setup``:
>> >
>> > * ``_uuid``, Android has no uuid/uuid.h header.
>> > * ``grp`` some grp.h functions are not declared.
>> > * ``_crypt``, Android does not have crypt.h.
>> > * ``_ctypes`` on x86_64 where all long double tests fail (`bpo-32202`_)
>> > and
>> > on
>> >   arm64 (see `bpo-32203`_).
>> >
>> > .. [1] On Wikipedia `Android version history`_ lists the correspondence
>> > between
>> >    API level, commercial name and version for each release. It also
>> > provides
>> >    information on the global Android version distribution, see the two
>> > charts
>> >    on top.
>> >
>> > .. _`README`:
>> > https://github.com/xdegaye/cpython/blob/bpo-30386/Android/README.rst
>> > .. _`bpo-26865`: https://bugs.python.org/issue26865
>> > .. _`bpo-30386`: https://bugs.python.org/issue30386
>> > .. _`bpo-32202`: https://bugs.python.org/issue32202
>> > .. _`bpo-32203`: https://bugs.python.org/issue32203
>> > .. _`PR 1629`: https://github.com/python/cpython/pull/1629
>> > .. _`buildmaster-config PR 26`:
>> > https://github.com/python/buildmaster-config/pull/26
>> > .. _`Android version history`:
>> > https://en.wikipedia.org/wiki/Android_version_history
>> > .. _`termux`: https://termux.com/
>> > .. _`termux-packages`: https://gitlab.com/jbwhips883/termux-packages
>> > .. _`adb`: https://developer.android.com/studio/command-line/adb.html
>> > .. _`Unified headers`:
>> >
>> > https://android.googlesource.com/platform/ndk.git/+/ndk-r14-release/docs/UnifiedHeaders.md
>> > .. _`HP 41`: https://en.wikipedia.org/wiki/HP-41C
>> > .. vim:filetype=rst:tw=78:ts=8:sts=2:sw=2:et:
>> > _______________________________________________
>> > Python-Dev mailing list
>> > Python-Dev at python.org
>> > https://mail.python.org/mailman/listinfo/python-dev
>> > Unsubscribe:
>> >
>> > https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/brett%40python.org