Bytecode optimisation

[Martijn Faassen]

Jim Meier wrote:
 
> The thread on "swallow" seems to have dried up; maybe they've gone out to
> experiment with it?

I haven't *exactly* experimented yet, though I'm thinking about the C
generating part of Swallow now. Basically Swallow (internally) could
look like this:

Python code (in Swallow subset) + Static type info (for instance in
dicts) ->
Python parse tree with type annotations (error if type annotations are
missing or conflicting).

Now, from that Python parse tree with type annotations (in fact a
Swallow parse tree) we need to go to a 'parse' tree that can generate C
code. Note that the C tree doesn't have to be able to generate any
construct possible in C, just the parts necessary for Swallow.

I think the tricky parts are adding the type info to Swallow parse tree
(along with the type info for lots of builtin stuff), and then turning
this tree into the C tree.

Perhaps someone already has a python module that can generate C code
from some tree structure?

Random note: I think we need a form of C++ templates or generic
classes/types in order to do enough of Swallow. For instance, we need a
list(integer) and list(float) and list(fooClass). Some system of generic
programming seems necessary here.

> In any case, these ideas and the ideas from that thread are
> butting head-on. If a satisfactory way of finding "static" functions and
> type-constant variables, then that markup could be used both to optimise the
> python byte-codes and to generate native code directly.

Swallow in my view initially depends on manually adding this static type
info (and then Swallow assumes it's correct!).

> Depending on the speed
> of the system and how often the optomisation is done, this could be done
> automatically in the interpreter.

Not if you have to add static type info manually. Of course type
inference engines can always be added to Swallow later. :)
 
> It may be nescessary to keep two or more copies of a function around (if you
> want both speed and flexibility).. one optimized to flash powder in native
> code, one in the original python bytecode.

Yes, I had the same idea.

> When doing it's lookups, a quick
> second check for a) known types and b) if the types are known, then if they
> match the types assumed in the optimisation. If the check passes, it runs the
> optimised version. If not, it runs the standard version.
> 
> (this might mean that only "swallowed" code can call other "swallowed" code
> without wrappers. I don't think this is a big deal?)

No, this sounds like a good idea.

Regards,

Martijn