Mailman 3 October 2014 - Python-ideas

Adding iOS/Android support to Python
by Russell Keith-Magee Oct. 27, 2014

Oct. 27, 2014

Hi all, tl;dr - I'd like to add iOS and Android to the list of platforms that Python supports out-of-the-box, and I'm looking for guidance on the way forward (and assistance, if anyone is keen). Longer version: For the last 9 months or so, I've been working on tools to enable developers to write native mobile apps using Python. My motivation is to get Toga (http://pybee.org/toga) to the point where it's a viable way to write mobile apps. However, I'm not alone in the "Python on mobile" goal - Kivy has been around for a few years, with similar goals (although they've got some different ideas and priorities). To date, I've been using the groundwork established by Kivy, which takes the Python 2.7.1 (for iOS) and Python 2.7.2 (for Android) source code, applies a bunch of patches, and wraps those in a set of build scripts to yield Python libraries that can be used on iOS and Android. I'd like to start supporting more recent versions of Python - most importantly, Python 3. While I could certainly continue maintaining patches externally (and I imagine I'll have to if I want to maintain Python 2.7 support), I'd like to see that effort integrated into Python's repository. In particular, there are four areas where I can see changes required: 1) At least 2 new sys.platform definitions - "ios" and "android" (a third "iossimulator" platform might also be needed - the iOS simulator has enough differences that it can be helpful to be able to identify the platform) 2) Modifications to the build system to support cross-platform builds for mobile - AFAICT, cross platform builds currently only work on Linux and require readelf; iOS libraries can only be built on Mac, and building Android libraries don't require readelf (AFAICT). 3) Disabling certain modules on mobile platforms. Supporting modules like linuxaudiodev, ossaudiodev, readline, curses, idle and tkinter on mobile platforms doesn't make much sense; modules likes bsddb and bz2 are difficult to support due to library dependencies; and the need for modules like multiprocessing is arguable (and difficult to support on mobile). Even providing a Python executable/shell is arguable for these platforms. 4) Support for building fat libraries for iOS. Xcode uses a tool called lipo to mash multiple platform libraries into a single library file. I don't imagine 1 & 2 are especially controversial. However, I would anticipate that 3 might raise some concerns, as the general Python "batteries included" philosophy would be modified to "unless you're on one of these platforms". Any guidance on whether this approach would be palatable? Is there any precedent for "module not supported on platform X" that I'm not aware of? Any platforms where Python is *only* available as a library? I have no idea how to tackle 4. To create a complete iOS build, you have to do at least 4 complete Python builds - a native system build (so you can run Python scripts that are part of the build), an i386 build (for the simulator), an armv7 build, and an arm64 build - and then the build products of the last three need to be combined into a single framework directory. Given three independent platform-specific build directories, it's relatively easy to construct a "fat" framework, but it isn't clear to me how that would fit into Python's build system. I can see that the build system has support for "universal" builds, which looks like an artefact of the OS X PPC->i386 conversion, but the build approach appears to be different to that required for iOS. Is there any documentation of the build process itself? Or is there anyone who can point me in the right direction? I'm also not clear how to tackle the testing and CI problems. Since you can't run iOS code on a non-iOS machine, it isn't entirely clear to me what an acceptance test would look like for a mobile build. So - am I completely wasting my time? Are patches for mobile platforms likely to be accepted into Python's repository? Or should I stick with the "external patches" approach? Any pointers, hints, guidance or assistance would be gratefully accepted. Yours, Russ Magee %-)

12 22

Fwd: Adding iOS/Android support to Python
by Russell Keith-Magee Oct. 26, 2014

Oct. 26, 2014

On Sat, Oct 25, 2014 at 3:36 PM, Todd <toddrjen(a)gmail.com> wrote: > On Oct 25, 2014 4:22 AM, "Russell Keith-Magee" <russell(a)keith-magee.com> > wrote: > > 3) Disabling certain modules on mobile platforms. Supporting modules > like linuxaudiodev, ossaudiodev, readline, curses, idle and tkinter on > mobile platforms doesn't make much sense; modules likes bsddb and bz2 are > difficult to support due to library dependencies; and the need for modules > like multiprocessing is arguable (and difficult to support on mobile). Even > providing a Python executable/shell is arguable for these platforms. > > I would definitely be extremely interested in a python shell in android. > One thing I feel are lacking on android are good advanced mathematical > tools and and python shell with appropriate libraries could make a very > powerful open-source tool for that. There have been some attempts at that > already. > Yes - and (to the best of my knowledge) none of them provide the default Python shell. They're custom user interfaces, using native system controls, that provide a shell-like UI. What I'm talking about here is the literal "python.exe" build target - the one that is an executable that starts and expects to attach to stdin/stdout. *That* executable isn't practical on Android *or* iOS, because neither platform has the concept of a "console" in the traditional Unix sense of the word. > I would also differentiate android and iOs more. Android seems to be > betting on multi-core performance while iOs seems to be betting on > single-chore performance. So while multiprocessing may not make much sense > on iOs, I think it may be more sense on Android, especially if they move > from 4 to 8 cores. > Firstly - I don't know what gave you the impression Apple devices aren't multicore - every Apple processor since the A5 (introduced in the iPhone 4S and iPad 2) has been at least dual core, and the A8X in the newest iPads is triple core. Secondly, if you're assuming "multicore" automatically means "mathematical powerhouse", you're mistaken. If you're planning on doing serious mathematical heavy lifting on a phone... well, you've already made your first mistake :-) > Similarly, ultimately android is Linux-based, so things like readline > should work, and it seems others have been able to get it to work, and bz2 > seems to work fine on third-party android file managers. > iOS is Posix under the hood as well. My point was that readline and curses make a whole lot of assumptions about the type of terminal you're using, and if you're writing a halfway decent UI for a mobile platform, you'd probably be better served throwing those tools overboard in preference for something that takes native UI controls into account. As for bz2 - I can't think of any reason it wouldn't work on Android (or iOS either, for that matter); I was more flagging the fact that it has binary dependencies which need to be resolved, and external dependencies complicate the cross-compilation process. More broadly, as a result of both these points, I was trying to gauge the extent to which "complete availability of all Python modules" would be considered a requirement of any official effort to port Python to Android and iOS. Yours, Russ Magee %-)

2 2

Fwd: Adding iOS/Android support to Python
by Russell Keith-Magee Oct. 25, 2014

Oct. 25, 2014

On Sat, Oct 25, 2014 at 9:27 PM, Hernan Grecco <hgrecco(a)gmail.com> wrote: > > El oct 24, 2014 11:22 PM, "Russell Keith-Magee" <russell(a)keith-magee.com> > escribió: > > > > Hi all, > > > > tl;dr - I'd like to add iOS and Android to the list of platforms that > Python supports out-of-the-box, and I'm looking for guidance on the way > forward (and assistance, if anyone is keen). > > Have you looked a QPython? They provide Python 2 and 3 consoles in > android. I am not sure how large is the stdlib support, I have only tried > small scripts and it works quite nicely. > Yes, I've tried it, but I'm in a similar boat in that I didn't exhaustively exercise the full stdlib. However, iOS and Android are both Posix. I'm sure *many* of the extension modules - especially the ones that are purely data-based, like bz2 - could be compiled. However anything with a UI element (like tkinter) doesn't make any sense. Yours, Russ Magee %-)

1 0

namespace for backported stdlib modules?
by Robert Collins Oct. 21, 2014

Oct. 21, 2014

I want to make the improvements in unittest (which transitively includes mock and traceback and perhaps inspect ...) available for folk that can't use 3.5 yet. We've currently got the unittest2 and mock as modules on pypi, and clearly we can add another e.g. traceback2 as a way to get the traceback changes out there. I'm wondering though if perhaps putting up a namespace package on pypi for things in the stdlib - e.g. 'stdlib' (or stdlib2 or something) would make sense. In particular that might permit relative imports to automatically pick up the other also backported modules without source changes. So we'd have e.g. stdlib.unittest stdlib.traceback stdlib.mock And then using that would be done through the normal try: from stdlib import unitest except ImportError: import unittest dance. -Rob -- Robert Collins <rbtcollins(a)hp.com> Distinguished Technologist HP Converged Cloud

9 12

Need feedback about exposing an operator overloading in a module
by Julien Palard Oct. 20, 2014

Oct. 20, 2014

Hi everybody, Long long time ago, I wrote thehttps://pypi.python.org/pypi/pipe Python module, TL;DR: a 6 lines of code decorator enabling infix syntax. Long time ago someone came to speak to me on IRC advising me to speak about it here, but I was still blocked in my mind about the exposed/leaked `|` operator overload. For me, a module should never expose an operator overload, because it may lead to unpredicted/unnatural/surprising behavior. But I may be wrong. Recently I was thinking about my pipe module and got an idea: What about exposing a single function instead of a whole bunch of operator-overloaded functions ? Like Pipe (because I can't spot a real name for the moment but need one for the example) and translate from: fib() | pp.where(lambda x: x % 2 == 0) | pp.take_while(lambda x: x < 4000000) | pp.add to Pipe(fib()).where(lambda x: x % 2 == 0) .take_while(lambda x: x < 4000000) .add().data It's a bit more dense but still far more readable than the classic suffix notation: pp.add(pp.take_while(lambdax: x < 4000000, pp.where(lambda x: x % 2 == 0, fib()))) (I explicitly prefixed by `pp.` to explicityly show that using the `|` operator we have to import either the module or every used functions, but using Pipe we do not have to) For clarity, where are the two implementations (Doctests permiting you to compare usability of both): Actual Pipe module: https://github.com/JulienPalard/Pipe/blob/master/pipe.py Pipe module with a function instead of operator overloading: https://github.com/JulienPalard/Pipe/blob/less_intrusive/pipe.py So a few questions: 1) Do you think, like me, that exposing a `|` overload is bad ? 2) In the less intrusive version, do you think, like me, that .register, Pipe(...), and .data are too much ? 3) Do you want to see something permitting infix notation in Python standard library ? Bests, -- Julien Palard

4 3

Query about proposed Multi-line Anonymous Function syntaxes
by Adam Jorgensen Oct. 14, 2014

Oct. 14, 2014

Hey all, my first time posting here. So, as I understand it, the main reason we don't have multi-line anonymous functions is because there has been no consensus about the syntax. I was just wondering if someone has a document detailing the various syntaxes that have been proposed in the the past. Adam

8 9

Make len() usable on a generator
by Thomas Chaumeny Oct. 12, 2014

Oct. 12, 2014

Hi! I have just come across some code counting a generator comprehension expression by doing len([foo for foo in bar if some_condition]) and I realized it might be better if we could just use len(foo for foo in bar if some_condition) as it would avoid a list allocation in memory. Another possibility is to write sum(1 for foo in bar if some_condition), but that is not optimal either as it generates a lot of intermediate additions which should not be needed. Sure, len(generator) might lead to an infinite loop but since sum(generator) is allowed in Python I see no reason why len(generator) isn't. What do you think ?

20 38

Negative limit for traceback.*
by Thomas Allen Oct. 11, 2014

Oct. 11, 2014

I'd like to propose adding the ability to extract last n stack trace entries from a traceback using functions from traceback module. Simply put, passing limit=-n would make functions to return (or print or yield) last abs(n) entries. No-brainer implementation for extract_tb: extracted = list(_extract_tb_iter(tb, limit=(limit, None)[limit < 0])) if limit is not None and limit < 0: return extracted[limit:] return extracted The motivation: http://stackoverflow.com/q/25472412/2301450

3 3

reduce(func, seq, initial=0)
by Stephan Sahm Oct. 10, 2014

Oct. 10, 2014

Hi folk, I stumbled upon the builtin reduce function. It has an optional parameter initial, however if trying to set it like import operator as op reduce(op.add, range(10), initial=0) I get TypeError: reduce() takes no keyword arguments I think it would be really straightforward to also add keyword possibilities, at least to the parameter initial. What do you think? best, Stephan

6 8

Extend unicodedata with a name search
by Philipp A. Oct. 10, 2014

Oct. 10, 2014

I noticed that the excellent perl utility unum <http://www.fourmilab.ch/webtools/unum/> uses an obsolete unicode database. Since I’m a Pythonista, i recalled hearing about the stdlib unicodedata module, using which I either wanted to rewrite unum or extend its database. Unfortunately, unicodedata is very limited. Partly rightfully so, since you can convert codepoints and chars with chr() and ord(), and str.upper() and friends are unicode-aware. But the name database is only queryable using full names! I want to do unicodedata.search('clock') and get a list of dozens of glyphs with names like CLOCKWISE RIGHTWARDS AND LEFTWARDS OPEN CIRCLE ARROWS and CLOCK FACE THREE-THIRTY. Maybe this should spit out a list of (name, char) tuples? or a {name: char} dict? What do you mean?

8 15