[Edu-sig] FYI: PataPata postmortem link

[Paul D. Fernhout]

Ian-

Thanks for the comment.

I think the issue you raise is a really difficult one to resolve. If I 
understand the general issue correctly, when you are looking at a GUI 
interface with underlying processes, and a person wants to "see the code", 
the question is what to show them, and also whether to show it to them in 
such a way as they can modify either what they see right then, or what 
they might see if the restarted the program.

One thing I had sometimes dreamed about is mapping every pixel on the 
screen with a programming or debugging equivalent of a "Z-Buffer". But in 
this, call it perhaps an "S-Buffer", instead of storing Z-depth from a 
pixel on a polygon, you could associate a stack trace with every pixel 
from the process that last wrote to that pixel (you might even store the 
stacks of everyone who wrote to that pixel). So, in that way you would 
have a somewhere logical to begin to explore the system related to 
constructing each aspect of the current view. If you see a button on a 
screen, mouse over to it, hit a key, and immediately you are in the 
debugger with a stack trace related to the code that drew the button, 
which presumably would link to the button itself. You would need pretty 
much the entire stack since otherwise you would just end up with 
references to the polygon drawing routines. :-) Possibly not feasible with 
today's technology though -- you might need several K per pixel at least, 
and with a megapixel display, that might mean gigabytes just for that 
storage. Might be possible in a few years though.  Or might be feasible 
sooner for beginners if you restricted the window to, say, a 200 X 200 
size, or if you could do something clever related to not storing 
independent copies of each stack trace, since whole areas of pixels would 
likely be drawn by essentially the same common routines with the same 
underlying stack. Perhaps, considering that overlap, if you skipped the 
last one or two specific frames on the stack, you might be able to store 
such a "S-Buffer" with little more than an extra 10 megabytes or so for 32 
bit pointer for each pixel of a megabit display, plus maybe just a few 
megabytes for the really relevant stack? Something someone in academia 
might want to explore, perhaps as a student project, possibly. Of course, 
if you really wanted to do it right, you should save the state of the 
virtual machine in it's entirety when writing each pixel. So that is 
probably a few megabytes at least per pixel, or probably a few terabytes 
per display. Now I now what I'll be doing with all that memory the 
computer I buy in 2020 will have; storing a million similar copies off the 
same virtual machine. :-) Again though, with clever ideas like 
compression, storing checkpoints (like OCaml does) or using other 
approaches to VM design (the Pointrel System I wrote has a data storage 
approach in some versions which only adds data, never changes or erases 
it), it might be possible to reduce the memory demand for such a "complete 
VM state saving S-Buffer" down to only dozens or hundreds of megabytes per 
display, which is within reach even now. Even if this worked, you might 
still find exploring the system awkward if the system created objects in 
one place, but displayed them at another time in a refresh, as most 
systems do. You would still need then to work backward to figure out how 
the particular object on the screen got initialized. So maybe all you 
really need that is ultimately useful is an association of a graphical 
object doing th drawing with each pixel on the screen? And essentially, 
that is what PataPata does (and many other drawing-composition-oriented 
programs as well); it maps from a pixel on the screen back to a GUI 
widget, and then presents that widget in a browser. Anyway, I think 
perhaps there are some genral ideas to explore here.

Whether changes to anything you explored by picking a pixel from the 
screen were useful to modifying what you saw right then and there would 
have a lot to do with the underlying programming model. In the case of 
conventional Python GUIs, where you have a builder method that assembles 
all the widgets, it would really take a restart of the GUI (or at least 
the modified window) to map back from source to display. And any changes 
just made to widgets would not likely be easily saved with that model -- 
unless you pursued writing out object to source in some fashion similar to 
what PataPata tried. The notion of a Smalltalk image (a saved binary file 
representing the current state of a world of object) seems a much easier 
approach to support this kind of dynamic interface redesign, and I think 
that is one of the reasons I feel more productive programming GUIs in 
Smalltalk than Python; I rarely have to restart a complex Smalltalk 
application the way I often have to do with Python ones (unless you use 
various tricks, some of which I have posted on before to the Jython list).

While I did end up doing things "out of process" a lot in terms of editing 
the source file as text, I think the reasons I did it on further 
reflection could have had more to do with my confidence about the system 
then anything else. I was often making changes to key GUI components or 
other core implementation things (including file writing). If I was 
working as a "user" level instead of a "maintainer" level, I think the "in 
process" editing of windows using the hierarchical browser might have been 
a lot more satisfactory and appealing. But that remains speculation; still 
I think when I used the inspector to change things, I probably was 
thinking more like I was testing the system from a "user" standpoint.
Of course, the whole point of something like PataPata is to blur the line 
of "user" and "maintainer", so the issue is never that simple.

Thanks again for your conceptual help with key ideas in the project like 
using metaclasses to build prototypes; that part of going from source to 
prototypes I was very pleased with. :-)

--Paul Fernhout

Ian Bicking wrote:
> Paul D. Fernhout wrote:
> 
>>Just as an FYI, as a way to wind up the PataPata project (or at least one 
>>phase of it), I wrote a lengthy postmortem critique of the PataPata 
>>project to date, plus ideas for where to go from here. You can read the 
>>critique by following this link:
>>
>>"PataPata critique: the good, the bad, the ugly"
>>http://sourceforge.net/mailarchive/forum.php?thread_id=31111569&forum_id=48729
>>
>>Comments welcome.
> 
> 
> Thanks for putting this together; it's useful to know how this went, as 
> it was a fairly ambitious rethinking of Python development.
> 
> There's some similar issues being considered for OLPC.  Specifically the 
> laptop has a "view source" key, which (unsurprisingly) lets you view the 
> source of what you are doing.  What that means is application (aka 
> "Activity") specific, but a general Python solution is called for.
> 
> Of course it's a little hard to say what that actually should be.  Just 
> a text editor?  Not too exciting.  Really there should be a view of the 
> process.  And some way to manipulate the in-process objects and source. 
>   One thing I was unsure of -- and I think your postmortem confirmed -- 
> is whether it's really feasible to generally edit objects in-process in 
> a persistent way.  I'm inclined not to go so far, perhaps simply relying 
> on UI hints to indicate when the change is likely to be persistent. 
> It's fairly easy to edit a function definition persistently, for 
> instance, or add a function definition.  You can still explore and poke 
> around in the process any way you want, but you don't have complete power.
> 
> Of course, at some point you need complete power, which means editing 
> the source in ways that can only be meaningful when you restart the 
> process from scratch.  I'm not sure how or where to make that break. 
> OTOH, emphasizing clean/fast/easy restarts might be more general and 
> easier to implement than in-process persistent editing.
> 
> I'm also not sure what to build on, or what to use.  It's interesting 
> that you felt out-of-process interaction was successful.  This fits with 
> my own intuition that in-process stuff can be dangerous and fragile. 
> But I'm not that familiar with the details of how the interprocess 
> communication should happen; perhaps IDLE is a good place to start 
> looking.  Perhaps IDLE is a good place to start the development?  I 
> don't believe Tk will be available on the laptops, only GTK, and 
> probably not wx either, so that limits the possibility of reusing most 
> UI, though other bits of code might be fine.
>