[Edu-sig] More OLPC chatter (PataPata; Edit&Continue)

Paul D. Fernhout pdfernhout at kurtz-fernhout.com
Sun Jul 1 15:27:04 CEST 2007


First, to emphasize a key idea below, everyone here should see for a six
minute video walkthrough example:
  "A nice demo of Smalltalk development in the debugger by David Buck."
  http://motionobj.com/article/demo-of-smalltalk-visualworks
  http://www.simberon.com/smalltalkdemo.avi
That's what OLPC should ideally be able to do with a "View Source" key IMHO.

====

Well, we've been round this before, obviously:
  http://www.mail-archive.com/pypy-dev@codespeak.net/msg02442.html

Developing a more dynamic way to view and edit source on running systems
built on Python was, of course, the main point of the PataPata
experiments. :-)
  http://patapata.sourceforge.net/
Here is Francois Schnells' related demo video:
  http://showmedo.com/videos/video?name=patapata_tkinter1_fSchnell

The phrase that now seems to be in vogue to describe such capabilities
is "Edit and Continue". Not quite entirely the same, but it covers
enough to be really useful as a search term. This feature is of great
value to beginners when it is integrated into an IDE well, as it helps
them immediately focus on broken pieces of code rather than finding them
by more indirect means.

In hindsight, I wish I had focused more on getting plain Python and
Jython to be more friendly in terms of "Edit and Continue":
  http://www.google.com/search?hl=en&q=edit+and+continue
like by hacking the core C and Java code, and perhaps IDLE and PyDev,
rather than focusing on using plain-vanilla Python to support
prototypes, to add properties, or wrapping various widget libraries.
After all, Smalltalk (Squeak or VisualWorks or several others) all allow
"edit and continue" even though they are, like Python, mostly class
based. In the end your (Kirby's) points have proven all too valid --
it's hard to get a new paradigm accepted, especially when it goes
against the grain of how underlying libraries are written, all the books
out there, and so on. The path of least resistance really would have
been to add an "edit and continue" patch to C-Python (or Jython),
perhaps centered around a PEP. It's not what I wanted -- since I wanted
backward compatibility with older Python versions -- but in the end it I
feel it would have been more effective, and likely a better payback for
the person-months of effort that went into the PataPata experiment.
Still, that's why people do experiments, to learn, and I learned. :-)
Hopefully others can benefit from what I learned too.

I still feel one thing PataPata got right (as a technical advance,
inspired by Self) was showing how you could save and restore the state
of a world of Python objects (and their behaviors) just by writing a
plain text (and highly readable) Python program which reconstructed that
world of objects (with that made slicker by using Ian Bicking's
suggestions for meta class uspport). See my "postmortem" self-critique
(for more information:
  "PataPata critique: the good, the bad, the ugly"
  http://patapata.sourceforge.net/critique.html
I feel that's a good idea to keep in the back of your mind when thinking
about OLPC applications and "view source". And it's so much nicer to do
things that way than via pickle which produces an almost unreadable (and
so effectively unmodifiable) output of a live set of Python objects.

On "edit and continue", from Microsoft's web site:
  http://msdn2.microsoft.com/en-us/library/bcew296c.aspx
"Edit and Continue is a time-saving feature that enables you to make
changes to your source code while your program is in break mode. When
you resume execution of the program by choosing an execution command
like Continue or Step, Edit and Continue automatically applies the code
changes with some limitations. This allows you to make changes to your
code during a debugging session, instead of having to stop, recompile
your entire program, and restart the debugging session."

I pick Microsoft's definition even though they were late to the party
(compared to Lisp and Smalltalk doing this decades ago) just to show how
mainstream such technology has become. Isn't this omission just a
glaring embarrassment for Python at this point? And no, module reloading
doesn't cut it, as it entails building your own hooks into your
application as well as thinking through a lot of implications of what
that means -- definitely not something for Python newbies to consider
doing. I myself add limited support for module reloading in a commercial
application while developing and it makes a lot of difference when
developing an application which has long load times or long processing
times -- but it's not a trivial thing to manage, nor is it the same as
"Edit and continue".

So I think another attempt towards PataPata's ideals, but not
emphasizing prototypes, and considering changes to the Python and Jython
cores (C and Java), might be worth trying from a Python community point
of view -- or even as Python3000 feature. Sadly, I'm now wrapped up in
other things and can't devote even a few weeks (or more) to realizing
this right now; plus I do think it is so central to the system it would
take a while (for me) to get up to speed on C-Python internals and
probably even longer to get such potentially radical changes accepted if
submitted from someone as a first patch to C-Python.

I know everyone tries to get Guido to add his or her own pet feature to
Python, and so am I, :-) but look, if this was a fairly straightforward
independent change like a bug fix or just a library add-on, anyone could
do it; even though the code involved is small, this takes a true snake
wrangler to get sure this is done right, and an overall commitment by
the Python leadership to make sure it keeps working right as other
changes are made to Python in the future. There are several issues, but
the most important ones probably revolve around better support within
the Python library for linking code to a place in a file (some parts of
that are broken, like for one liner functions), within an IDE like IDLE
building on that to allow fine-grained browsing of changes to just a
single function in version control, and within the Python core dealing
with issues with more easily allowing functions to be modified without
producing new function pointers. Restartable exceptions would be ideal
for this, but even without them, this could be doable -- just have the
system break on creating certain specified exceptions before throwing them.

I think it's a shame that Python can't more easily support this sort of
"edit and continue" development; it already has most of the hooks. Just
a few specific almost trivial things it lacks -- and I think it lacks
them more from a conservative mindset than anything else -- an emphasis
that restarting a program when you make a small change or hacking each
one individually to reload modules is somehow still OK in the 21st
century. :-) I'd simply hate to think this related comment by someone
else was true:
  http://www.mail-archive.com/pypy-dev@codespeak.net/msg02446.html
"you stand a better chance of doing this in pypy than the python
developers do. they consider it "impossible", because it's too radical
a rethink, because they have been trapped, by design, in c-code."

Anyway, Smalltalk, Lisp, and other languages (including now Visual Basic
and most languages from that vendor) have been doing this for decades. I
hope OLPC can be the impetus to overcome this glaring omission with
Python; otherwise, the OLPC group may find their entire approach to
using Python instead of, say, Lisp or Smalltalk or Ruby or several other
languages for the core was a mistake. Still, I'm sure at this point it
would be much easier for the OLPC project to fix Python than to switch.
Well, I guess OLPC could stick with Python as is, but the View Source
key will always be a bit of a disappointment, as it won't be an "Edit
and Continue" style "View the source you you can edit the running
program like Squeak does and see your changes work immediately" sort-of
key, but more a "View the source and that's it for now, maybe you can
restart later" kind of key.

This issue is painted by you (Kirby's) as "maturity" -- that kids are
someday mature enough to use something like, say, Microsoft
VisualStudio. I think that, while true in general that programmers need
to learn to use a variety of tools as they mature, it is misleading in
this particular situation -- considering how other systems like Lisp or
Smalltalk have been doing "edit and continue" for decades -- allowing
you to *easily* modify a running program and continue. The obvious issue
IMHO is "immaturity" -- there is a piece of Python or its community
culture which is lacking. And once Python gets that feature as part of
its continuing maturation process, watch out, because after adding easy
"Edit & Continue" internal support, and then using that support from an
IDE like IDLE or PyDev, then Python will grow overnight to 10X the
creature it is already! :-)

Not to bash Python too much here; I use Python despite its glaring
shortcoming in this area because it is so wonderful in many other
important ways (licensing, libraries, readable syntax, helpful
community, cross-platform support, namespaces, etc.) which in practice
often outweigh this lack of "edit and continue". I mainly just want to
see Python get even better -- especially considering the time and
learning and code investment I have in it.

Rather than agree with the comment above about being locked into a
corner by using C, I could speculate there is too much of a C developer
"culture" holding out there somehow (which is perfectly fine most of the
time as Python is written in C, and C can be a great language if used
well. It's just that thinking in C culture terms (C culture --> edit,
compile, start, run, crash, debug, stop, edit, compile, restart, repeat)
just hurts a lot in this one particular area (where Smalltalk culture ->
start, crash, debug, crash, debug, crash, etc. when you edit code in the
debugger).

Somehow I feel if Guido could just see, say, VisualWorks (or ideally
VisualWorks+ENVY) in action for an hour by an expert (not Squeak, which
has too many other ideas overloaded on it), he'd see what was really
possible and desirable with dynamic objects. Gee, maybe I should Google
that? Hmmm... Well what do you know! :-) Here is a six minute example:
  "A nice demo of Smalltalk development in the debugger by David Buck."
  http://motionobj.com/article/demo-of-smalltalk-visualworks
  Links to:
  http://www.simberon.com/smalltalkdemo.avi
The video above shows the cycle of "Debug, Crash, Debug, Crash, etc."
while incrementally developing a Solitaire-style card game, except a
crash is really just an exception thrown and launches the debugger,
where the problem is fixed, and you're able to continue. How can anyone
watch this six-minute video of VisualWorks and not want Python (and
OLPC) to be able to do the same thing and just as easily? (Note the
debugger, browser, inspector, and selected code evaluation all being
used together -- that's really what it takes to do good incremental
coding -- but it is also almost all it takes. :-)

Still, I do think there is a fundamental issue when thinking about "view
source" key in interaction with a GUI, since many GUIs are generated by
a function, so it is always potentially unclear whether you are
modifying the instance of the GUI you are looking at or are modifying
the method which generates that GUI. I think that issue is approachable,
but it does add layers of conceptual complexity. Prototypes in PataPata
resolved some of that conceptual complexity (as live prototypes could
defined the interface), but at a cost of adding complexity to Python in
terms of support libraries and using Python in a non-standard way which
made problems harder to debug. So, by moving away from prototypes and
back to classes (including using the PythonCard-like approach for
building GUIs hung off a central class holding all the behavior of
individual widgets instead of a PataPata approach where widgets had
significant behavior stuck in them), one needs to rethink how to handle
that approach. One generic thought for the far future I had on that was
that each pixel on the screen should be tried to a stack trace as the
code was when that pixel was written, however, even that might miss the
link to where the objects performing the behaviors in question were
initialized. In that sense, I think Smalltalk's approach of supplying a
code browser, inspector, and debugger, and with a developer typically
using all three at the same time as a team to figure out where objects
of interest were initialized, is really the way to go (and also
evaluating snippets of code from any handy text pane).

Still, I think you (Kirby) are right in general on the issue of a
multiplicity of editors for various situations. And I think that is a
good thing for people to eventually learn. However, specifically, the
reality is that most behavior of interest (in terms of changing) on
something like the OLPC (especially to novices) is likely to be coded in
Python. This also shows the value of Jython and Java for Python
development on other systems, as it would be easy to make a system that
lets you tinker with both levels (Python=slow but smart code, Java=fast
but dumb code) since Java has better reflective capabilities than C;
still C could be compiled in debug mode to make such information more
available, and even run-time modifiable by invoking the compiler and
some sort of linker for modified functions. Though also, in the
VisualWorks case, most Smalltalk code typically runs about 6X slower
than the speed of C or so, which is generally good enough (rather than
Python sometimes approaching 100X slower for loops and motivating more C
coding, and thus better C tool integration).

And your (Kirby's) emphasis on multiple editors suggest this idea: one
way to approach the "View Source" key on the OLPC project that might be
to treat "view source" as a "meta key" and perhaps press other keys at
the same time to get at various aspects of the running program --
P=process list, G=GUI widgets, M=underlying modules, I=initialization
code, L=command line, T=current thread, S=source code manager,
B=breakpoints (including on certain E=exceptions), B=loaded binary
modules (and associated source and editors like if in C), and so on.

--Paul Fernhout

kirby urner wrote:
> Here's a floating email fragment
> 
>> Date: Thu, 28 Jun 2007 20:54:09 -0400
>> From: "Walter Bender" <walter.bender at ...>
>> Subject: OLPC software: our first release and beyond.
>> To: "OLPC Devel" <devel at ...>, sugar at ...
> 
>> One important feature where we have made little progress is the View
>> Source key. We decided to build the user experience?and the core
>> Activity base?in Python, in large part, to facilitate the children's
>> direct access to modifiable code. We are moving to the latest stable
>> version of Python as part of our Fedora Core 7 migration; however, we
>> have yet to put enough resources into building a suitable development
>> environment for children. This remains an important goal, but not one
>> we can reasonably meet for our first release. The incorporation of the
>> context-sensitive spirit of "view source" into all Activities is
>> another area where we lag. Bolstering these efforts is second only to
>> stabilizing the current system. We look forward to the possibility of
>> Guido van Rossum, Python's creator, leading these efforts in the fall.
> 
> Always interesting to see where some people are hoping to invest
> some of our BDFL's precious clock cycles...
> 
> Stepping back for a moment, one has to realize this is a different
> shop talk, as we typically look at source code as text files, modifiable
> yes, but not while running as binaries, which are different files entirely,
> derived from said text files by various processes.
> 
> Some of this text is expressed in pure Python, but a lot of what
> *runs* under Python in modular form is actually written in something
> else, as is Python itself.  A whole different text editor might be more
> appropriate in that case, or at least different plug-ins (the Eclipse model).
> 
> The trend is to target a shared VM and play freely with other language
> communities, swapping whatever works in inter-pluggable arrangements.
> 
> One way around this shortcoming of Sugar's is to suggest that once
> kids are mature enough to eyeball the kind of convoluted source
> code required to keep the likes of Sugar alive, it might be time to
> transition to a more standard model laptop that doesn't push the
> bleeding edge so hard.
> 
> Adults need to focus on stable developer environments like Wing's
> or whatever (or like IDLE for starters).  So at your bat mitzvah or
> other ceremony, you maybe trade in your much loved OLPC XO
> (and your teddy bear), and move up to something more expensive,
> with a more traditional source code treatment (anyone ready for
> C yet?).
> 
> Or just start using more of those legacy TuxLab desktops, running
> a variety of distros (Ubuntu Studio might draw some kids, more
> because they want to study graphical arts than they want to do
> lexical programming).
> 
> Basically, I'm somewhat Kusasian in outlook:  go with immersive up
> to a point, then take a break and study some classic combos e.g.
> emacs + bash + gcc.  Drop all the way down to machine language
> even, coming up through MMIX enroute to Python Part Deux (or
> "snake guts revisited").[1]  At some Coming of Age level, we no longer
> want to shield kids from the messy reality of *no* single governing
> paradigm at all levels and the old immersive environment turns out
> to be one fish tank among many.
> 
> In retrospect, those early experiences with special namespaces
> for children will remain a comfort (we hope), and some will elect
> to professionally teach them to a next generation, and/or invent
> new ones (already the case, so a next iteration).
> 
> Kirby
> 
> [1]  http://worldgame.blogspot.com/2007/06/techie-chatter.html


More information about the Edu-sig mailing list