[Python-Dev] Re: pirate (python+parrot)

[Michal Wallace]

On Mon, 4 Aug 2003, Dan Sugalski wrote:

> To be honest, I'd rather leave it as it is--not because I don't
> think that parsing python and compiling to parrot isn't worthwhile
> (I think it is) but because I can be reasonably assured that the
> conversion will take place (because I want it to happen and have
> wagered pie on that want, and thus will make it happen :). Besides,
> I'm not much of a parser guy, thus the parsing part's actually more
> difficult than the bytecode translation, which is reasonably
> straightforward.

Fair enough, but have you looked at pirate? The 
parser's done. Python exposes its own parser via
the parser module. There's a python module for
walking that tree, and all I'm doing is calling
that and writing methods.

The only potential snag in the scheme is the exec 
statement, because somehow parrot's got to have a 
python parser. But unless I'm missing something
completely, you have the same problem with your
bytecode interpreter because there is no way 
you can do exec without a python parser. :) 

So, since the rules as they are say nothing about
not having exec in there, Guido can just build all
kinds of dynamic python code strings and test out
the exec statement, and you're either going to have
to parse it or get pied. :)

On the other hand, there's already at least one 
seemingly-working python parser written in python:

   http://codespeak.net/moin/pypy/moin.cgi/BytecodeCompiler

To me, the shortest path looks like this:

   - use python's parser to build the tree
   - finish the pirate code generator
   - wrap or recode PyObjects as PMC for the built-in types
   - wrap the python parser library or use a pyPython parser
     so we can get eval working without the python source
   - focus on optimizing the generated bytecode

I know you're more comfortable looking at bytecode, 
and yes, some of the translation stuff might be 
applicable to translating other stack-based bytecode
to parrot, but it just seems like it's not really 
helping anyone. 

I read through Klaas-Jan's entire Lua->parrot paper
yesterday: http://members.home.nl/joeijoei/parrot/report.pdf
and except for the for statement (which has different
semantics in python), our generated IMCC and the problems
we're dealing with are almost identical. That's why
we're talking about merging the code generators.

I think compiling python source to parrot bytecode is 
going to take a couple weeks AT MOST. It would go a
whole lot faster if someone like you or Leo were
pairing with me. Then you could make sure it's spitting
out good parrot bytecode (and I wouldn't have to spend
hours hunting through the docs), and I could make sure the
python semantics are doing the right thing (and you wouldn't
have to worry about parsing python). We'd both move a
whole lot faster.

Really, pair with me for an hour or two and we can knock
out function definitions or something. I think you'll see
it's a piece of cake. We just both ssh to my server and
use screen and an IRC or AIM window to chat, and I'll
walk you through the python stuff and you can show me
the IMC/PIL/PIR/PASM. It would fly by.

Then the tricky part is just making the bytecode run as
fast as possible, and dealing with the built-in types,
which is what you probably really want to be doing, right?

Honestly, the goal shouldn't be to be faster than the
python virtual machine. Your goal should be to be to 
deliver something so compelling that the python team votes 
unanimously to dump the current VM in favor of parrot
for python 3.0. :) Of course if they ever did that,
an old-bytecode to parrot-bytecode interpreter wouldn't
be of much use.

On the OTHER hand, a good generic parrot compiler that's
benefitted from Dan the Parrot God's expertise is going
to be valuable to *anyone* who wants to compile their
language down to parrot. It helps PHP, it helps Lua.
It helps Ruby. It could even help Perl! :) 

Long term, I'm imagining a situation where people don't 
really have to deal with code generation at all, except
to override the parts that need to be customized for
their own language. It'll boil down to generating
an AST and then transforming the tree to fit with the
generic compiler's worldview, and then hitting "go". 
Everything from that point on down - peephole optimization,
calling conventions, imcc, would be standard. 

Okay, I'm rambling here, so I'll stop... But take me
up on my pairing offer. Give me an hour or two. Then,
if you're not convinced, fine. And if you are, I'm 
definitely committed to seeing this through to the
end.

Sincerely,
 
Michal J Wallace
Sabren Enterprises, Inc.
-------------------------------------
contact: michal at sabren.com
hosting: http://www.cornerhost.com/
my site: http://www.withoutane.com/
--------------------------------------