[Edu-sig] FYI: PataPata postmortem link
Paul D. Fernhout
pdfernhout at kurtz-fernhout.com
Tue Nov 28 04:12:42 CET 2006
Thanks for the comment.
I think the issue you raise is a really difficult one to resolve. If I
understand the general issue correctly, when you are looking at a GUI
interface with underlying processes, and a person wants to "see the code",
the question is what to show them, and also whether to show it to them in
such a way as they can modify either what they see right then, or what
they might see if the restarted the program.
One thing I had sometimes dreamed about is mapping every pixel on the
screen with a programming or debugging equivalent of a "Z-Buffer". But in
this, call it perhaps an "S-Buffer", instead of storing Z-depth from a
pixel on a polygon, you could associate a stack trace with every pixel
from the process that last wrote to that pixel (you might even store the
stacks of everyone who wrote to that pixel). So, in that way you would
have a somewhere logical to begin to explore the system related to
constructing each aspect of the current view. If you see a button on a
screen, mouse over to it, hit a key, and immediately you are in the
debugger with a stack trace related to the code that drew the button,
which presumably would link to the button itself. You would need pretty
much the entire stack since otherwise you would just end up with
references to the polygon drawing routines. :-) Possibly not feasible with
today's technology though -- you might need several K per pixel at least,
and with a megapixel display, that might mean gigabytes just for that
storage. Might be possible in a few years though. Or might be feasible
sooner for beginners if you restricted the window to, say, a 200 X 200
size, or if you could do something clever related to not storing
independent copies of each stack trace, since whole areas of pixels would
likely be drawn by essentially the same common routines with the same
underlying stack. Perhaps, considering that overlap, if you skipped the
last one or two specific frames on the stack, you might be able to store
such a "S-Buffer" with little more than an extra 10 megabytes or so for 32
bit pointer for each pixel of a megabit display, plus maybe just a few
megabytes for the really relevant stack? Something someone in academia
might want to explore, perhaps as a student project, possibly. Of course,
if you really wanted to do it right, you should save the state of the
virtual machine in it's entirety when writing each pixel. So that is
probably a few megabytes at least per pixel, or probably a few terabytes
per display. Now I now what I'll be doing with all that memory the
computer I buy in 2020 will have; storing a million similar copies off the
same virtual machine. :-) Again though, with clever ideas like
compression, storing checkpoints (like OCaml does) or using other
approaches to VM design (the Pointrel System I wrote has a data storage
approach in some versions which only adds data, never changes or erases
it), it might be possible to reduce the memory demand for such a "complete
VM state saving S-Buffer" down to only dozens or hundreds of megabytes per
display, which is within reach even now. Even if this worked, you might
still find exploring the system awkward if the system created objects in
one place, but displayed them at another time in a refresh, as most
systems do. You would still need then to work backward to figure out how
the particular object on the screen got initialized. So maybe all you
really need that is ultimately useful is an association of a graphical
object doing th drawing with each pixel on the screen? And essentially,
that is what PataPata does (and many other drawing-composition-oriented
programs as well); it maps from a pixel on the screen back to a GUI
widget, and then presents that widget in a browser. Anyway, I think
perhaps there are some genral ideas to explore here.
Whether changes to anything you explored by picking a pixel from the
screen were useful to modifying what you saw right then and there would
have a lot to do with the underlying programming model. In the case of
conventional Python GUIs, where you have a builder method that assembles
all the widgets, it would really take a restart of the GUI (or at least
the modified window) to map back from source to display. And any changes
just made to widgets would not likely be easily saved with that model --
unless you pursued writing out object to source in some fashion similar to
what PataPata tried. The notion of a Smalltalk image (a saved binary file
representing the current state of a world of object) seems a much easier
approach to support this kind of dynamic interface redesign, and I think
that is one of the reasons I feel more productive programming GUIs in
Smalltalk than Python; I rarely have to restart a complex Smalltalk
application the way I often have to do with Python ones (unless you use
various tricks, some of which I have posted on before to the Jython list).
While I did end up doing things "out of process" a lot in terms of editing
the source file as text, I think the reasons I did it on further
reflection could have had more to do with my confidence about the system
then anything else. I was often making changes to key GUI components or
other core implementation things (including file writing). If I was
working as a "user" level instead of a "maintainer" level, I think the "in
process" editing of windows using the hierarchical browser might have been
a lot more satisfactory and appealing. But that remains speculation; still
I think when I used the inspector to change things, I probably was
thinking more like I was testing the system from a "user" standpoint.
Of course, the whole point of something like PataPata is to blur the line
of "user" and "maintainer", so the issue is never that simple.
Thanks again for your conceptual help with key ideas in the project like
using metaclasses to build prototypes; that part of going from source to
prototypes I was very pleased with. :-)
Ian Bicking wrote:
> Paul D. Fernhout wrote:
>>Just as an FYI, as a way to wind up the PataPata project (or at least one
>>phase of it), I wrote a lengthy postmortem critique of the PataPata
>>project to date, plus ideas for where to go from here. You can read the
>>critique by following this link:
>>"PataPata critique: the good, the bad, the ugly"
> Thanks for putting this together; it's useful to know how this went, as
> it was a fairly ambitious rethinking of Python development.
> There's some similar issues being considered for OLPC. Specifically the
> laptop has a "view source" key, which (unsurprisingly) lets you view the
> source of what you are doing. What that means is application (aka
> "Activity") specific, but a general Python solution is called for.
> Of course it's a little hard to say what that actually should be. Just
> a text editor? Not too exciting. Really there should be a view of the
> process. And some way to manipulate the in-process objects and source.
> One thing I was unsure of -- and I think your postmortem confirmed --
> is whether it's really feasible to generally edit objects in-process in
> a persistent way. I'm inclined not to go so far, perhaps simply relying
> on UI hints to indicate when the change is likely to be persistent.
> It's fairly easy to edit a function definition persistently, for
> instance, or add a function definition. You can still explore and poke
> around in the process any way you want, but you don't have complete power.
> Of course, at some point you need complete power, which means editing
> the source in ways that can only be meaningful when you restart the
> process from scratch. I'm not sure how or where to make that break.
> OTOH, emphasizing clean/fast/easy restarts might be more general and
> easier to implement than in-process persistent editing.
> I'm also not sure what to build on, or what to use. It's interesting
> that you felt out-of-process interaction was successful. This fits with
> my own intuition that in-process stuff can be dangerous and fragile.
> But I'm not that familiar with the details of how the interprocess
> communication should happen; perhaps IDLE is a good place to start
> looking. Perhaps IDLE is a good place to start the development? I
> don't believe Tk will be available on the laptops, only GTK, and
> probably not wx either, so that limits the possibility of reusing most
> UI, though other bits of code might be fine.
More information about the Edu-sig