[pypy-dev] MinimalPython newbie questions

[Bengt Richter]

At 15:34 2003-01-21 +0100, Armin Rigo wrote:
>Hello Sébastien,
>
>On Tue, Jan 21, 2003 at 11:33:12AM +0100, Sébastien Pierre wrote:
>> Looking at the <www.tunes.org> site, which is maintained by Armin,
>
>I'm not the maintained of that site, althought I often cite it.
>
>> My question is: will MinimalPython try to incorporate the microthreads 
>> from stackless and will try to incorporate ideas from tunes project 
>> (like migration) ? If so MinimalPython would really be an ideal 
>> language for developing agent-based applications.
>
>Maybe.  There are quite a lot of exciting possibilities that we can dream of,
>given enough flexibility.  Starting from a high-level description of the
>interpreter (the interpreter written in Python), Stackless microthreads can
>easily be added (or not).

I'm wondering how you will get to the starting point of the
"high-level description" you mention by any automated means
of translating the interpreter from C sources. I.e., you could
even say
    repr(file('/usr/bin/python','r').read())
gets you a quick "Python source representation" of a big chunk
of Python, but it isn't high level. Reading in the C sources into
Python strings gets you another representation held in Python's hands
(or coils ?). But your goal is to transform this into another kind of
representation of the same abstract thing, but at a "high level," so
that the latter will be suitable for (a) being transformed again by
Psyco to an efficient low level for a CPU, and (b) for humans to use
as a new master representation of the language interpreter, suitable
for human modifications and experimentation.

(a) could be viewed as a black box super-optimization project accepting
C as the ongoing master source representation, and it wouldn't really
matter what the intermediate representations were in generating an
efficient fast interpreter. That might be an interesting thing, but
I am guessing that is not the key goal for pypy-dev.

b) seems a difficult thing, because the C code does not _directly_ reflect
the higher level abstractions you want to re-represent in Python. Stepping
back for a moment, what would the ideal Python source for Python look like?
I don't think it would look like a mechanical translation from C.

IMO the concern of keeping up with CPython sources should be dropped in
favor of keeping up with developing/running a comprehensive test suite
for all versions purporting to implement the same Python language version.

The C sources are really irrelevant as far as the language is concerned.
Plus there will be times when CPython develops bugs that don't happen
when a new PEP is implemented in PPython (hm, handy short name ;-), and
vice versa (until PPython replaces CPython as the official master ;-)

Except insofar as you want to keep C as the master representation of
some functionality, and hope to gain by black-box super-optimization
via Psyco without human intervention, or perhaps want to use C as source
and not use an external C compiler for some reason, I suspect that automated
translation of C should best be used only to generate a starting point
for human editing. If C remains a master representation for something,
it may be that an ffi approach is more practical until Python source is
available.

And an editing starting point doesn't even have to be compilable, though it
doesn't hurt to make it so through quick-and-dirty comment passthroughs
of C for human reference, and sticking to otherwise easy stuff to get some
useful tool going fast.

Hm, I have a general feeling that in the game of programs-as-data
you are probably doomed to reinvent some lisp ;-)

Set your time machine to 1960 for minimal language-in-language inspiration ;-)
http://lib1.store.vip.sc5.yahoo.com/lib/paulgraham/jmc.lisp

A really high level PPython source is probably nice for humans, but a
somewhat lower level, using a Python language subset might be easier
for Psyco to chew on? So are you planning on identifying stages of
language features, and partitioning the PPython source, where the
lowest level boostraps higher level capabilities to use for the next
stage, or are there only two levels, so the entire language has to be
implemented in the lower level? How are you dealing with the temptation
to code before designing ;-) (I recognize you have to find some things
out by doing, and "plan to throw one away" etc., but ... ;-)

>Migration is a whole harder project, althought if we know what we precisely 
>want it will probably be possible to add it then without rewriting everything 
>(which is the whole point of the MiniPython project).
>
>Migration is related to checkpoints, and saving and restoring program state.  
>Scott pointed out numerous problems related to it.  I personally believe that
>these problems araise from some kind of confusion between the various
>abstraction levels; when you want a checkpoint, you cannot make a copy of the
>state of the whole universe, so you must be clear about which abstraction
>levels you want to save and restore.  For example, from the C point of view, a
>program can only be checkpointed between two expressions; from the processor
>point of view, it can be checkpointed between two instructions; but the level
>that probably interests you is higher.  In a GUI application you may want that
>the checkpoint occurs between two elementary user actions.  This is a whole
>complex problem, outside the scope of MiniPy (which could however become later
>a nice test bed).

I agree with your point re abstraction levels, but OTOH, ISTM the design
of MiniPy/PPython will affect whether you _can_ do larger-granularity
checkpointing without resorting to instruction-level checkpointing.
Also different kinds of checkpointing are feasible depending on when
in the development of execution state you call for it (e.g., before
opening any user files, or starting threads, etc.) You could prohibit
a lot and still have a useful restricted capability.