[Mobile-sig] python on android

[Russell Keith-Magee]

Hi Frank,

Thanks for the tips. I'll try these out and see how I go.

Yours,
Russ Magee %-)

On Fri, Jan 30, 2015 at 1:53 AM, Frank, Matthew I <matthew.i.frank at intel.com
> wrote:

>  I believe that I’ve seen this particular problem when building Python 3
> for Android, but that it comes and goes.  The current open patch for Python
> 3.5 that you need to apply is: http://bugs.python.org/issue22625.
>
>
>
> For Python 3 the problem is in Makefile.pre.in around line 750:
>
>
>
> $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGEN)
>
> Should be
>
> $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGENSRCS)
>
>
>
>
>
> For cross-building Python 3.4.2 from a Linux host I used the following
> configure command for the target build:
>
>
>
> CPPFLAGS=-I/abs/path/to/source/Python-3.4.2/FIXLOCALE
> ../Python-3.4.2/configure --enable-shared
> --prefix=/absolute/path/to/android/install/of/python
> --build=x86_64-linux-gnu --host=i686-linux-android --disable-ipv6
> ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no
> ac_cv_little_endian_double=yes --without-ensurepip
>
>
>
> The –without-ensurepip, in particular was necessary because ensurepip
> needs to call the just-built Python interpreter (which won’t work when
> cross compiling).
>
>
>
> There are some other things that need patching for Android (around the mbs
> to wcs functions and nl_langinfo() and such), but that’s the only problem
> with the Makefile that I’m aware of.
>
>
>
> -Matt
>
>
>
> *From:* Guido van Rossum [mailto:guido at python.org]
> *Sent:* Thursday, January 29, 2015 9:54 AM
> *To:* Russell Keith-Magee
> *Cc:* mobile-sig
> *Subject:* Re: [Mobile-sig] python on android
>
>
>
> My only suggestion: try to understand *exactly* what the Makefile is doing
> at that point.
>
> If it's really just invoking the wrong Python binary to do the
> byte-compilation, you could also skip that until you're further along. The
> byte-compilation is optional (it slows startup down a bit, but you
> currently don't even start up, so who cares. :-)
>
>
>
> On Wed, Jan 28, 2015 at 10:58 PM, Russell Keith-Magee <
> russell at keith-magee.com> wrote:
>
>
>
> Ok - so, yes, libPython is just copying and byte compiling. The catch is
> which version of Python and Python library you use to do the compiling.
>
>
>
> To compile cross platform, you need to do 2 builds - a "host" build (x86
> for most desktop purposes), and then a "target" build (the architecture of
> your phone).
>
>
>
> (As an aside - in the case of iOS, you actually need to do 2 target builds
> - one for the simulator (which is x86, but with a different libc to OSX),
> and one for the device (which is an arm7/arm7s/arm64 triple target build) -
> but that's beside the point. The problems I'm having exist without this
> added complication).
>
>
>
> So you do the host build, then you do the target build. The target build
> needs to call Python to invoke the compilation of modules etc; so you need
> to invoke the host python, but use the setup.py that was configured for the
> target Python.
>
>
>
> This much I have working (at least, I think I do - I can't test yet
> because I don't have a working build, but all signs are positive).
>
>
>
> But then, you get to libinstall, and you need to invoke Python to byte
> compile the pyc files. To do this, you need to invoke the host python,
> using the host Python's library, but over the *target* Python's sources.
> It's this last step that is tripping me up at the moment - I haven't worked
> out how to drive Autoconf to drive configure to pass in the set of
> arguments to invoke Python so that it will use the right binary and library
> with the right library tree. I keep end up running the iOS binary (which
> doesn't start), or the x86 binary with the iOS library tree (which is
> missing some parts that Python needs).
>
>
>
> The problem manifests as an "ImportError: No module named _struct",
> because the compiled parts of the struct module aren't in the iOS tree (or,
> at least, aren't compiled for x86 in the iOS tree)
>
>
>
> I appreciate that this isn't the easiest problem to debug over a mailing
> list. It's not even strictly "debugging" - it's a matter of working out
> what combination of arguments I need to pass in, and working backwards to
> the automake script. I've already made a bunch of changes to get this far,
> and I'm guessing any suggestions would have to be informed by what I've
> already done, which isn't a trivial thing to communicate without dumping a
> huge patch on your lap and saying "hey, fix this for me".
>
>
>
> This is why, to date, I haven't sought out help - I've just been beavering
> through the problems one at a time :-) If you've got any suggestions on
> more productive ways to tackle this, I'm all ears. And, again, I'm happy to
> share what I've got to date if anyone is interested in helping out.
>
>
>
> Yours,
>
> Russ Magee %-)
>
>
>
> On Thu, Jan 29, 2015 at 2:17 PM, Guido van Rossum <guido at python.org>
> wrote:
>
>  Well, libinstall is nearly 100 lines, not counting dependencies. OTOH
> it's just copying a lot of files and then byte-compiling them into .pyc
> files -- and .pyc files are portable. So perhaps you can go into a little
> more detail?
>
>
>
> On Wed, Jan 28, 2015 at 9:10 PM, Russell Keith-Magee <
> russell at keith-magee.com> wrote:
>
>  Hi Guido,
>
>
>
> I'll glady stop arguing about Kivy vs Toga. The only reason I brought it
> up at all is because I keep hearing arguments that seem to dispute that
> getting a libPython build working *is* the first step. You've now put that
> argument to bed, so I agree - lets move on.
>
>
>
> For what it's worth, I've got a reasonable handle on how to compile
> libPython for mobile at this point - what I don't have is a good handle on
> is the intricacies of Python's build system, and in particular, how to
> drive Autoconf to support cross-platform builds.
>
>
>
> I've almost worked out the patches to the Python 2.7.1 source tree to
> generate an iOS-compatible libPython. Once I've got that working, I'm
> planning to merge those changes up to the tip of 2.7 and 3.X, and submit
> those patches for inclusion in the source tree. However, at the moment, I'm
> hitting problems with cross-platform execution in the libinstall target;
> I'm happy to share what I have so far with anyone interested in
> collaborating.
>
>
>
> Yours,
>
> Russ Magee %-)
>
>
>
> On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum <guido at python.org>
> wrote:
>
>  Can we stop arguing about Kivy vs Toga and focus on the one thing that
> they have in common, the need for a working Python 3 port on Android and
> iOS (for a start)? This is apparently mostly a matter of solving a lot of
> small things with the build system, dependencies, improved config files,
> and getting stuff integrated upstream so it can be built out of the box,
> right? After that the layers 2-4 stuff can compete, but everybody wins when
> layer 1 is dealt with (even imperfectly). It's pretty sad that nobody
> apparently knows how to reproduce the build steps, and everybody just
> copies the one Python 2.7.{1,2} binary that someone built out of sheer
> willpower.
>
>
>
> On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee <
> russell at keith-magee.com> wrote:
>
>
>
> Hi Bill,
>
>
>
> On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen <janssen at parc.com> wrote:
>
> Russell Keith-Magee <russell at keith-magee.com> wrote:
>
> >  b) I don't like the Kivy build tools. They're a lot more complex than
> they
> > need to be.
>
> I didn't find it troublesome, but of course this wasn't my first rodeo.
> I'd certainly agree it's not a push-button solution.  So, what would a
> less complex system be like?
>
>
>
> A less complex system is what Toga does.
>
>
>
>  1. You use cookiecutter to generate a stub project. This gives you the
> full source tree for a project you can load into XCode (iOS), or build with
> ant (Android), including a "hello world" __main__.py
>
>
>
>  2. You download the pre-compiled Python.framework for iOS, or libPython
> for Android, and copy it into a libs directory
>
>
>
>  3. You start writing Python code, replacing the __main__.py with your own
> logic.
>
>
>
>  4. You compile and deploy your project using XCode/ant.
>
>
>
> Compare this with Kivy - My experience was spending a couple of days
> getting the Kivy build process to actually work - trying to find versions
> of the Android NDK that aren't being distributed any more (but are the only
> hard coded options in the build system), working out that the provided code
> doesn't work with the most recent versions of Cython, and sourcing
> libraries for all sorts of dependencies, so that I could compile SDL and a
> bunch of OpenGL stuff - none of which I needed. It took me a couple of days
> to get to "hello world" - and all because of something that could have been
> shipped as a pre-compiled binary.
>
>
>
> > I'm going to guess the Kivy people are all Linux users, because
> > they don't appear to have worked out that binary compatibility is a
> thing.
>
> Sorry -- why is that a Linux thing?
>
>
>
> If I want to distribute an app for OS/X or Windows, I give you an
> executable, and It Just Works (tm). Source *might* be provided as an option
> in the interest of being open source, but it's not how you distribute
> anything in practice. The "Linux way" for distribution is to distribute
> source, and tell you to compile it yourself. Distributing binaries is an
> afterthought, because ABI compatibility makes building and distributing
> binaries painful. Most projects don't have the infrastructure to distribute
> binaries for multiple platforms, so unless you can get the OS to provide a
> recent binary for you, you compile from source.
>
>
>
> I see reflections of that bias here. Even though ABI compatibility exists
> for both iOS and Android as platforms, Kivy chooses to distribute as source.
>
>
>
> > You don't need to have every user compile Python and the rest of the Kivy
> > stack - you can just ship a binary library, and it will work on every
> phone
> > with the same hardware (I know, because Toga does this. The Toga
> > bootstrapping process is "clone this repo, and copy this file". You could
> > reduce this to "clone this repo" if you were happy putting binary
> artefacts
> > into version control.)
>
> Sure.
>
> >  c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on
> Android;
>
> I believe the mobile platform packaging tools are still stuck at 2.7.
> Kivy 1.8+ will run on Python 3 on desktop, though.
>
>
>
> Yes - but in practice, the absence of Python 3 on Mobile means that Kivy
> on Mobile is Python 2 only, and an old version at that.
>
>
>
> > and if you build them on OSX, when you're on the device, they report
> > sys.platform as "darwin".
>
> Seems like a bug; I imagine you're suggesting that the Kivy build
> process should patch that file to return "android"?  Although I never
> know what to look at to get that platform info correctly -- this is a
> larger Python issue.
>
>
>
> Yes, it's a bug (or at least a missing feature). The build system patches
> that Kivy use doesn't introduce anything for targeted builds (i.e., using
> an x86 platform to compile ARM64 binaries - which is what you're doing when
> you compile for mobile), and doesn't provide a platform definition for
> mobile.
>
>
>
> And kivy.utils.platform seems to return the proper thing.
>
>
>
> So... instead of they've introduced their own way to get access to
> information that Python already has a standard way of providing.
>
>
>
> > Going back to my post
>
> For those of you following along at home, here's Jeff's list (I had to
> go and look it up).
>
>
>
> Jeff?
>
>
>
> >  1. A library build of Python
> >  2. Templates to stub out a working Python project
> >  3. Libraries to do bridge between native language environments and
> Python
> >  (for me, that's Rubicon)
> >  4. Libraries for utilising native system services (for me, that's Toga)
>
> > - I agree with Kivy on layer 1, and I was able to use
> > their build tools to bootstrap my own. However, I have very different
> > opinions on layers 2-4.
>
> Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it
> provides examples, and you're supposed to extrapolate from them.  I
> guess that's a form of template.  For #3, there's Kivy's "pyjnius" (to
> access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" (to
> access Objective-C via runtime reflection)
> (https://github.com/kivy/pyjnius).  For #4, as I said, there's "plyer"
> (https://github.com/kivy/plyer).
>
>
>
> I'd also include most of Kivy in #4 as well, because that's how they're
> tackling the widget issue.
>
>
>
> We can have a separate discussion sometime about Django vs. Tornado :-).
>
>
>
> So - a monkey knife-fight at dawn, then :-)
>
>
>
> Yours,
>
> Russ Magee %-)
>
>
>
> _______________________________________________
> Mobile-sig mailing list
> Mobile-sig at python.org
> https://mail.python.org/mailman/listinfo/mobile-sig
>
>
>
>
> --
>
> --Guido van Rossum (python.org/~guido)
>
>
>
>
>
>
> --
>
> --Guido van Rossum (python.org/~guido)
>
>
>
>
>
>
> --
>
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/mobile-sig/attachments/20150130/763a9639/attachment-0001.html>