[Mobile-sig] python on android

Thu Jan 29 07:58:14 CET 2015

Ok - so, yes, libPython is just copying and byte compiling. The catch is
which version of Python and Python library you use to do the compiling.

To compile cross platform, you need to do 2 builds - a "host" build (x86
for most desktop purposes), and then a "target" build (the architecture of
your phone).

(As an aside - in the case of iOS, you actually need to do 2 target builds
- one for the simulator (which is x86, but with a different libc to OSX),
and one for the device (which is an arm7/arm7s/arm64 triple target build) -
but that's beside the point. The problems I'm having exist without this
added complication).

So you do the host build, then you do the target build. The target build
needs to call Python to invoke the compilation of modules etc; so you need
to invoke the host python, but use the setup.py that was configured for the
target Python.

This much I have working (at least, I think I do - I can't test yet because
I don't have a working build, but all signs are positive).

But then, you get to libinstall, and you need to invoke Python to byte
compile the pyc files. To do this, you need to invoke the host python,
using the host Python's library, but over the *target* Python's sources.
It's this last step that is tripping me up at the moment - I haven't worked
out how to drive Autoconf to drive configure to pass in the set of
arguments to invoke Python so that it will use the right binary and library
with the right library tree. I keep end up running the iOS binary (which
doesn't start), or the x86 binary with the iOS library tree (which is
missing some parts that Python needs).

The problem manifests as an "ImportError: No module named _struct", because
the compiled parts of the struct module aren't in the iOS tree (or, at
least, aren't compiled for x86 in the iOS tree)

I appreciate that this isn't the easiest problem to debug over a mailing
list. It's not even strictly "debugging" - it's a matter of working out
what combination of arguments I need to pass in, and working backwards to
the automake script. I've already made a bunch of changes to get this far,
and I'm guessing any suggestions would have to be informed by what I've
already done, which isn't a trivial thing to communicate without dumping a
huge patch on your lap and saying "hey, fix this for me".

This is why, to date, I haven't sought out help - I've just been beavering
through the problems one at a time :-) If you've got any suggestions on
more productive ways to tackle this, I'm all ears. And, again, I'm happy to
share what I've got to date if anyone is interested in helping out.

Yours,
Russ Magee %-)

On Thu, Jan 29, 2015 at 2:17 PM, Guido van Rossum <guido at python.org> wrote:

> Well, libinstall is nearly 100 lines, not counting dependencies. OTOH it's
> just copying a lot of files and then byte-compiling them into .pyc files --
> and .pyc files are portable. So perhaps you can go into a little more
> detail?
>
> On Wed, Jan 28, 2015 at 9:10 PM, Russell Keith-Magee <
> russell at keith-magee.com> wrote:
>
>> Hi Guido,
>>
>> I'll glady stop arguing about Kivy vs Toga. The only reason I brought it
>> up at all is because I keep hearing arguments that seem to dispute that
>> getting a libPython build working *is* the first step. You've now put that
>> argument to bed, so I agree - lets move on.
>>
>> For what it's worth, I've got a reasonable handle on how to compile
>> libPython for mobile at this point - what I don't have is a good handle on
>> is the intricacies of Python's build system, and in particular, how to
>> drive Autoconf to support cross-platform builds.
>>
>> I've almost worked out the patches to the Python 2.7.1 source tree to
>> generate an iOS-compatible libPython. Once I've got that working, I'm
>> planning to merge those changes up to the tip of 2.7 and 3.X, and submit
>> those patches for inclusion in the source tree. However, at the moment, I'm
>> hitting problems with cross-platform execution in the libinstall target;
>> I'm happy to share what I have so far with anyone interested in
>> collaborating.
>>
>> Yours,
>> Russ Magee %-)
>>
>> On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum <guido at python.org>
>> wrote:
>>
>>> Can we stop arguing about Kivy vs Toga and focus on the one thing that
>>> they have in common, the need for a working Python 3 port on Android and
>>> iOS (for a start)? This is apparently mostly a matter of solving a lot of
>>> small things with the build system, dependencies, improved config files,
>>> and getting stuff integrated upstream so it can be built out of the box,
>>> right? After that the layers 2-4 stuff can compete, but everybody wins when
>>> layer 1 is dealt with (even imperfectly). It's pretty sad that nobody
>>> apparently knows how to reproduce the build steps, and everybody just
>>> copies the one Python 2.7.{1,2} binary that someone built out of sheer
>>> willpower.
>>>
>>> On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee <
>>> russell at keith-magee.com> wrote:
>>>
>>>>
>>>> Hi Bill,
>>>>
>>>> On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen <janssen at parc.com>
>>>> wrote:
>>>>
>>>>> Russell Keith-Magee <russell at keith-magee.com> wrote:
>>>>>
>>>>> >  b) I don't like the Kivy build tools. They're a lot more complex
>>>>> than they
>>>>> > need to be.
>>>>>
>>>>> I didn't find it troublesome, but of course this wasn't my first rodeo.
>>>>> I'd certainly agree it's not a push-button solution.  So, what would a
>>>>> less complex system be like?
>>>>>
>>>>
>>>> A less complex system is what Toga does.
>>>>
>>>>  1. You use cookiecutter to generate a stub project. This gives you the
>>>> full source tree for a project you can load into XCode (iOS), or build with
>>>> ant (Android), including a "hello world" __main__.py
>>>>
>>>>  2. You download the pre-compiled Python.framework for iOS, or
>>>> libPython for Android, and copy it into a libs directory
>>>>
>>>>  3. You start writing Python code, replacing the __main__.py with your
>>>> own logic.
>>>>
>>>>  4. You compile and deploy your project using XCode/ant.
>>>>
>>>> Compare this with Kivy - My experience was spending a couple of days
>>>> getting the Kivy build process to actually work - trying to find versions
>>>> of the Android NDK that aren't being distributed any more (but are the only
>>>> hard coded options in the build system), working out that the provided code
>>>> doesn't work with the most recent versions of Cython, and sourcing
>>>> libraries for all sorts of dependencies, so that I could compile SDL and a
>>>> bunch of OpenGL stuff - none of which I needed. It took me a couple of days
>>>> to get to "hello world" - and all because of something that could have been
>>>> shipped as a pre-compiled binary.
>>>>
>>>>
>>>>> > I'm going to guess the Kivy people are all Linux users, because
>>>>> > they don't appear to have worked out that binary compatibility is a
>>>>> thing.
>>>>>
>>>>> Sorry -- why is that a Linux thing?
>>>>>
>>>>
>>>> If I want to distribute an app for OS/X or Windows, I give you an
>>>> executable, and It Just Works (tm). Source *might* be provided as an option
>>>> in the interest of being open source, but it's not how you distribute
>>>> anything in practice. The "Linux way" for distribution is to distribute
>>>> source, and tell you to compile it yourself. Distributing binaries is an
>>>> afterthought, because ABI compatibility makes building and distributing
>>>> binaries painful. Most projects don't have the infrastructure to distribute
>>>> binaries for multiple platforms, so unless you can get the OS to provide a
>>>> recent binary for you, you compile from source.
>>>>
>>>> I see reflections of that bias here. Even though ABI compatibility
>>>> exists for both iOS and Android as platforms, Kivy chooses to distribute as
>>>> source.
>>>>
>>>>
>>>>> > You don't need to have every user compile Python and the rest of the
>>>>> Kivy
>>>>> > stack - you can just ship a binary library, and it will work on
>>>>> every phone
>>>>> > with the same hardware (I know, because Toga does this. The Toga
>>>>> > bootstrapping process is "clone this repo, and copy this file". You
>>>>> could
>>>>> > reduce this to "clone this repo" if you were happy putting binary
>>>>> artefacts
>>>>> > into version control.)
>>>>>
>>>>> Sure.
>>>>>
>>>>> >  c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on
>>>>> Android;
>>>>>
>>>>> I believe the mobile platform packaging tools are still stuck at 2.7.
>>>>> Kivy 1.8+ will run on Python 3 on desktop, though.
>>>>
>>>>
>>>> Yes - but in practice, the absence of Python 3 on Mobile means that
>>>> Kivy on Mobile is Python 2 only, and an old version at that.
>>>>
>>>>
>>>>> > and if you build them on OSX, when you're on the device, they report
>>>>> > sys.platform as "darwin".
>>>>>
>>>>> Seems like a bug; I imagine you're suggesting that the Kivy build
>>>>> process should patch that file to return "android"?  Although I never
>>>>> know what to look at to get that platform info correctly -- this is a
>>>>> larger Python issue.
>>>>>
>>>>
>>>> Yes, it's a bug (or at least a missing feature). The build system
>>>> patches that Kivy use doesn't introduce anything for targeted builds (i.e.,
>>>> using an x86 platform to compile ARM64 binaries - which is what you're
>>>> doing when you compile for mobile), and doesn't provide a platform
>>>> definition for mobile.
>>>>
>>>>
>>>>> And kivy.utils.platform seems to return the proper thing.
>>>>>
>>>>
>>>> So... instead of they've introduced their own way to get access to
>>>> information that Python already has a standard way of providing.
>>>>
>>>>
>>>>> > Going back to my post
>>>>>
>>>>> For those of you following along at home, here's Jeff's list (I had to
>>>>> go and look it up).
>>>>
>>>>
>>>> Jeff?
>>>>
>>>>
>>>>> >  1. A library build of Python
>>>>> >  2. Templates to stub out a working Python project
>>>>> >  3. Libraries to do bridge between native language environments and
>>>>> Python
>>>>> >  (for me, that's Rubicon)
>>>>> >  4. Libraries for utilising native system services (for me, that's
>>>>> Toga)
>>>>>
>>>>> > - I agree with Kivy on layer 1, and I was able to use
>>>>> > their build tools to bootstrap my own. However, I have very different
>>>>> > opinions on layers 2-4.
>>>>>
>>>>> Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 --
>>>>> it
>>>>> provides examples, and you're supposed to extrapolate from them.  I
>>>>> guess that's a form of template.  For #3, there's Kivy's "pyjnius" (to
>>>>> access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus"
>>>>> (to
>>>>> access Objective-C via runtime reflection)
>>>>> (https://github.com/kivy/pyjnius).  For #4, as I said, there's "plyer"
>>>>> (https://github.com/kivy/plyer).
>>>>
>>>>
>>>> I'd also include most of Kivy in #4 as well, because that's how they're
>>>> tackling the widget issue.
>>>>
>>>> We can have a separate discussion sometime about Django vs. Tornado :-).
>>>>>
>>>>
>>>> So - a monkey knife-fight at dawn, then :-)
>>>>
>>>> Yours,
>>>> Russ Magee %-)
>>>>
>>>> _______________________________________________
>>>> Mobile-sig mailing list
>>>> Mobile-sig at python.org
>>>> https://mail.python.org/mailman/listinfo/mobile-sig
>>>>
>>>>
>>>
>>>
>>> --
>>> --Guido van Rossum (python.org/~guido)
>>>
>>
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/mobile-sig/attachments/20150129/0864a1c5/attachment-0001.html>