The way to a faster python [was Python IS slow !]

[Markus Kohler]

Martijn Faassen <faassen at pop.vet.uu.nl> writes:

> Markus Kohler wrote:
> 
> [how to speed up Python - interpreter optimizations and JIT]
> 
> I've been thinking about something else. I'm not quite sure how feasible
> this is, though. The idea goes something like this:
> 
> * Subset Python; make a strict subset so that any code written for the
> subset can still run in Python proper. Mainly remove some of the
> dynamism.
> 

This is the way squeak a free Smalltalk implementation 
 works (www.squeak.org). The complete virtual machine (
the interpreter) including the garbage collector is written in a small subset
of Smalltalk which  is automatically translated to C. This has the big 
advantage that one can debug the whole VM withing squeak !

> * Add static typing info; initially by hand though some type inference
> can always be added later:
> 
> import StaticTypes
> 
> __types_foo = { "bar"  : StaticTypes.int,
>                 "baz"  : StaticTypes.int,
>                 Result : StaticTypes.int }
> 
> def foo(bar, baz)
>     return bar + baz
> 
> __types_hoi = { "message" : StaticTypes.string }
> 
> def hoi(message):
>     print "Hoi", message
> 
> __types_printlist = { "mylist" : StaticTypes.list(StaticTypes.string),
>                       "i"      : StaticTypes.string }
> 
> def printlist(mylist):
>     for i in mylist:
>         print i
> 
> __types_test = { "a" : StaticTypes.int }
> 
> class Test:
> 
>     __types_init = { "a" : StaticTypes.int }
>     def __init__(self, a):
>         self.a = a
> 
>     __types_get_a = { Result: StaticTypes.int }
>     def get_a(self):
>         return self.a
> 
> __types_useTest = { "arg" : StaticTypes.Class("Test"),
>                     Result: StaticTypes.int } 
> def useTest(arg):
>     return arg.get_a()
> 
> Note that this is all still perfectly executable by the Python
> interpreter.
> 
> * I propose we'd call this statically typed subset of Python 'Swallow',
> after the famous air speed velocity question. :)
> 
> * Parse the Swallow code with, for example, John Aycocks parsing
> framework (http://www.csr.UVic.CA/~aycock/python/). This can already
> parse Python. 
> 
> * Now combine this with the static typing info, and generate C code. The
> idea is that the Swallow subset is limited enough so that very efficient
> C code can be generated. Of course quite a lot of static typing info for
> the standard library is also needed to complete the process.

Good idea. 

> 
> * Also automatically generate interface C code for the places where the
> generated C code talks to Python proper and vice versa.
> 
> I know this is like Python2C, but with added static type info, and no
> attempt is made to translate all of Python; just a subset.
> 
> The idea is that much existing Python code is already swallow code, if
> type information is added. Lots more Python code isn't 'swallowable',
> and this can stay like it is. Development happens with the normal
> interpreter as Swallow code remains at all time executable by CPython;
> only when speed is desired people can add the type info and 'swallow'
> some of their functions. Basically it's a way to write C extensions in a
> Python subset, instead of in C. 
> 
> Later on we can try to add type inferencing and automatic swallowing of
> portions of code that are deemed swallowable.
> 
> I would imagine that a Swallow compiler would allow very significant
> speedups of code, with a minimum of developer hassle. A compiler of
> existing Python code would be even better, of course, but more difficult
> to do, a

Ans one big advantage is that one could debug the untranslated code,
if one would take care that all the is still executable python code.  
> 
> Of course all of this contains a lot of handwaving, but a least each
> step of the roadmap is readily imaginable. :)
> 
> Does Swallow seem useful to others? Any stumbling blocks that wreck this
> idea? I'd like to see some discussion; if people like the idea, perhaps
> we can even do some coding.
> 
> Regards,
> 
> Martijn


IMHO it's a great idea. 


Markus

-- 
Markus Kohler  mailto:markus_kohler at hp.com