Hi Mart I'm back with some news about wpython. I completed all the work that I was committed to do till the end of the year. I made a lot of changes to the code, that I'll report here. First, I added several conditional compilation sections that enable or disable almost every optimization I introduced into the project. Everything is controlled by a new include file, wpython.h, which holds a lot of #DEFINEs for each one of them. Every #DEFINE has a brief explanation, and some report an example with Python code disassembled, showing what happens. It can be useful both to document the code (also to access to the interested parts), and to let people test the effect of all optimizations. There are also a couple of #DEFINEs which are useful to enable or disable all superinstructions, or to make wpython work like CPython (with all new optimizations and superinstructions disabled). Full tracing support required a big effort, due to the missing SETUP_LOOP/POP_BLOCK instructions used in FOR_ITER blocks. It was a pain in the neck to let them work, but I think I have found a good solution fot it. If I remember correctly, Collin asked in the past about performance with testing enabled. I believe that speed is comparable to CPython, since I can trace FOR_ITER blocks enter/exit with very little time spent intercepting them; stack unrolling (for forward jumps cases) is fast too. Restoring Python object model required much of the work. I reverted all the changes that I made to many PyObjects, and just added some accessory code only to a few of them. There are no more hacks, and code is quite polite; only CodeObject required one line of code change in the hash function, to let it calculate hash correctly for the constants tuple (because it can hold lists and dictionaries now, which usally aren't hashable). Every file in Include/ and Objects/ that I modified has only 1 diff (except frameobject.c, for tracing code), so it's easy so see what is changed and the extra helper functions that I added to introduce lists and dictionaries in the consts tuple. In the meanwhile I've added a little optimization for lists and dictionaries used in for loops. Writing this: def f(): for x in ['a', 'b', 'c']: print x generates the following (word)code with the previous wpython: LOAD_CONST (['a', 'b', 'c']) DEEP_LIST_COPY GET_ITER FOR_ITER because ['a', 'b', 'c'] is a mutable object, and a copy must be made before using it. Now it'll be: LOAD_CONST (['a', 'b', 'c']) GET_ITER FOR_ITER So code is reduced and memory consumption too, because there's no need clone the list. The trick works only for lists and dictionaries that holds non-mutable objects, but I found it's a common pattern in Python code. I've also updated the source to the latest Python 2.x version, 2.6.4. All tests pass, both with Debug and Release code, on Visual Studio Express with 32 bit code (I can't compile 64 bits versions with it). There are only a few open issues. test_syntax.py required some changes in the doctest (adding full filesystem path) to let them pass correctly. It's really strange, but... works now! test_compile.py has 2 tests disabled in test_compile_ast: #['<forblock>', """for n in [1, 2, 3]:\n print n\n"""], #[fname, fcontents], that's because there's no support for constants (except Num_kind and Str_kind) in the current ASTs code. However code compiles well, except that it cannot make use of the new constant folding code. I haven't updated Doc/library/dis.rst, which is exactly the same of CPython. I'll do it when I stop introducing or changing opcodes. Right now wpython requires manual patching of Include/Python-ast.h, with the following lines: enum _expr_kind {BoolOp_kind=1, BinOp_kind=2, UnaryOp_kind=3, Lambda_kind=4, [...] List_kind=18, Tuple_kind=19, Const_kind=20}; enum _expr_const {no_const=0, mutable_const=1, content_const=3, pure_const=7}; struct _expr { enum _expr_kind kind; union { [...] struct { object c; enum _expr_const constant; } Const; } v; int lineno; int col_offset; }; They are need to let ast.c handle constants for the new constant folding code. I greatly appreciate any help to let it be generated automatically with ASDL grammar. That's all about the new code. Now the weird and stupid part. A few days I got a new gmail account, but accidentally I removed the google account that I've used to create the wpython at Google Code. I definitely lost project ownership, so I can't tag the old code and put the new one in trunk. I'll thank very much if someone that works or has contacts with Google can ask for moving ownership from my old account (cesare at pronto do it) to my new (the one which I've using now to write this mail), so I'll commit ASAP. Alternatively, I need to create a new project at Google Code. I hope that the community will appreciate the work (when I'll upload it :-). I know that it's a young project, but I think it's mature enough to take a look at it. Last but not least, think about it like a starting point. I have many ideas on how to optimize several other parts of Python, and the wordcode structure gives me rooms to do it in an elegant and efficient way thanks to the superinstructions (when needed). For the next release I plan to cleanup opcode.h and ceval.c, grouping some instructions into single superinstructions (CALL_FUNCTIONs and IMPORT_NAME), adding a few opcodes, and tweaking a bit the VM main loop (primarily targeted to reduce the jump-table that compilers produce for the big switch statement). Then I'll consider porting it to python 2.7 and/or python 3.1/3.2 if there'll be interest and feedbacks about it. I'm also at your disposal to discuss any detail about wpython source code, since I know that it isn't a simple patch to apply to some files. Cheers, Cesare 2009/11/4 Mart Sõmermaa

Thanks for the recap and for the good work on wpython!

Best, eagerly waiting for the results of your work to land in mainline python, MS