Pythoniac: Thoughts on a hardware Python processor

[Christopher Saunter]

Dear All,

I have seen the occasional post about creating a specific hardware 'Python
Processor', that could then be built and used in programmable logic or the
like (Valves, relays, core memory - Pythoniac ;-) - in other words, create
a processor that executes Python bytecode directly.  There would be
various different reasons for doing this (the 'neat' factor etc...)  

Replies to these questions have been a bit scant, so I though I would
chuck something in.

Due to Lazy Typing Syndrome I've used a couple of abbreviations:
PVM - Python Virtual Machine / Python Interpreter 
PBC - Python Byte Code

This has been done for Java, for example
http://www.xilinx.com/prs_rls/0134lavacore.htm 
and I have seen a partially complete open source core out there somewhere,
but I can't find it right now.

I have recently been looking at doing this for Python, and it seems like a
rather odd task.  From playing with the 'dis' module and reading some
docs, it seems that Python Interpreter (PVM), which would be translated
into a processor, has these properties:


o	Operates on sequential bytecode with branching / jump
instructions.
o	Entirely stack based  (no concept of registers / memory)

So far, so good - a processor could be built to do these functions of the
interpreter.  However, we go on....

o	Rather than operating on 8/16/xxx bit words, as most hardware
processors do, the PVM processes objects (via pointers on the stack),
which are treated in a 'black box' fashion - i.e. they have add, subtract,
compare etc. methods that are called by the PVM, and that return a result.
These methods of objects may not actually be coded for in Python, but in
the machine language (x86, alpha etc, derived from C etc.) of the
underlying processor.

Here, the PVM is very different from a 'typical' hardware processor, as
the later has a strictly limited number of data types, which it is able to
manipulate, whereas Python has many, built out of the basic types, and
although some may be coded in Python, they themselves are built on top of
C code (or Fortran etc.) that are all translated to native (x86 etc)
bytecode outside of Python.  The upshot of this is that the PVM is
unaware, in the majority of cases, of what actually needs to be done to
manipulate objects, it just makes calls to non Python code.

So, how to go about dealing with this in a Python processor?  

a) Create a stack based Python bytecode processor to replace the PVM, and
have this interact with another processor (x86 etc.) which stores the
objects and executes their methods. Ick.

b) Create a stack based bytecode processor that interfaces to custom
hardware processors for each Python object.  Ick.

Neither of these are particularly nice (or for b, workable) solutions.
The one I prefer is the following:

c) Create a processor that understands Python bytecode, using a stack for
the object pointers.  However, the processor would execute a sort of
superset of PBC, call it P+.  This would be fully compatible with Python,
but would include several built in types (32bit integers, 8bit integers
for strings etc,) and would have access to a memory space.  The basic
Python types(1) would then be implemented using P+, and stored in the
memory available to P+.  

c Should present a processor that executes Python bytecode, manipulating
the Python stack.  Then, when a method of an object is called, the
appropriate piece of P+ is called, executed by the same processor, and it
modifies the data structures of the objects, and the Python stack as
necessary, before returning seamlessly to the PBC execution.

Python extensions would then be directly executable on this processor, if
coded in Python.  If they are coded in C or some other language, a C->P+
compiler would be needed...

The level of abstraction caused by having to chase down function and
object pointers will be higher than for the execution of code on a more
conventional hardware processor, but this is also the case for executing
PBC with an interpreter.  

One advanced idea (not really thought about it yet!) is to store the
Python stack in main memory, but to store the top x (e.g. 32) items of the
stack in a register file to allow bytecode parallelism.  I gather that
IA-64 may do something sort of similar.

---

So, Pythoniac: a hardware (well, FPGA(2)...) processor that directly
executes Python bytecode.  What do people think about the idea?  Is it
really feasible?  I have doubts it would be particularly quick (in the
same way I have my doubts about anti-gravity generators... ;-), but it
should be fun!  

I would like to try and implement this, initially in a simulation
environment, then in a real FPGA, with hooks back to a PC allowing the
loading, execution, single step execution and monitoring etc of PBC.
Ideally I would like to reach a state where an interactive interpreter
runs over a serial port... 

So, anyone else interested in this?  Comments, suggestions, gaping holes
in my understanding of things as outlined above etc are welcome!

Regards,
	Chris Saunter



(1) Now we have increasing type<->object integration, should we refer to
numbers, strings, lists etc as types or objects?

(2) FPGA - Field Programmable Gate Array - a chip that consists of a
'logic fabric' than can be configured to produce arbitrary circuit
designs.  If this sounds new to you, try www.fpgacpu.org