Partial Parsing of Python

[Peter Eckersley]

On Sun, Jun 30, 2002 at 09:48:33AM +0200, holger krekel wrote:
 
> That's great! I can offer you code which will help you. 
> Moreover some of us had good discussions at EuroPython about *exactly*
> this type of project. For a start look at my alpha-version of a new 
> command line completer:
> 
>     http://home.trillke.net/~hpk/rlcompleter2.py
> 
> Michael Hudson moreover has done nice work with replacing readline
> with curses. 
> 
>     http://starship.python.net/crew/mwh/hacks/pyrepl.html

Excellent.  Thanks for the pointers....

> 
> Actually the problem goes deeper. Have a look at my TryParser class
> in rlcompleter2.py for a start. It does handle many cases and should
> give you some more ideas :-)

Agreed.  Although pyrepl seems about on par with what I'd tried,
TryParser seems to handle lots of trickier cases.  

> 
> Actually the REAL problem to solve first is to have Multi-Line editing. 
> Then all sorts of partial parsers *and* partial evaluation.  Python is
> just too damn dynamic to have that one easily.
> 

Oh, which reminds me of something else.  I also have problems with
side-effects ("heisenberg" effects)-- it will be necessary to get rid of
these.  My feeling was that this could be done using purity analysis and
return-type annotation.  Many of the important cases can be handled by
just adding information about the builtin modules, but it might also
be possible to do generalised python purity analysis.  Has anyone tried
this?

> So what to do? Our idea at EuroPython was to use PyGame (the wrapper
> around SDL) to get rid of the uglyness and restrictiveness of 
> readline and curses alltogether.  We want to have a REAL playground 
> and concentrate on our ideas.

Hmmm, I agree that one needs to escape readline/curses - but SDL sounds
like overkill, since I imagine you'll have to re-implement lots of basic
widgets on top of it.  I picked GTK, and have found it generally
suitable, despite a few quirks.  Having said that, there are still some
issues I need to resolve, so I shouldn't start counting chickens...

A copy of my (pre-pre-alpha) code is available at
http://www.cs.mu.oz.au/~pde/src/gush-0.003.tgz (it's currently called
gush - the Grand Unified SHell - although I think somebody might have
already used that name), so you can see how GTK works for this.  All
completions except obj.attr<cursor> are currently turned off.

In order to run it, you'll need the latest version of pygtk (1.99.10 or
later), from ftp://ftp.gtk.org/pub/gtk/python/v2.0/ .

I haven't dealt with multi-line editing yet, but GTK should certainly keep
this simple (just replace the GtkEntry for the command with a
GtkTextView).  I've been using up/down cursors to cycle through
completions, but they should really be used for the history, so adding
multi-line editing will make them triply-overloaded :)

> 
> Once i slow down from the mind-blowing EuroPython discussions and talks
> i check how easy that is.
> 
> Drop me a note if you are interested to stay tuned.
> 

Definitely...

-- 
Peter Eckersley
Department of Computer Science   &                  mailto:pde at cs.mu.oz.au 
IP Research Institute of Australia             http://www.cs.mu.oz.au/~pde
The University of Melbourne