First, to emphasize a key idea below, everyone here should see for a six minute video walkthrough example: "A nice demo of Smalltalk development in the debugger by David Buck." http://motionobj.com/article/demo-of-smalltalk-visualworks http://www.simberon.com/smalltalkdemo.avi That's what OLPC should ideally be able to do with a "View Source" key IMHO. ==== Well, we've been round this before, obviously: http://www.mail-archive.com/pypy-dev@codespeak.net/msg02442.html Developing a more dynamic way to view and edit source on running systems built on Python was, of course, the main point of the PataPata experiments. :-) http://patapata.sourceforge.net/ Here is Francois Schnells' related demo video: http://showmedo.com/videos/video?name=patapata_tkinter1_fSchnell The phrase that now seems to be in vogue to describe such capabilities is "Edit and Continue". Not quite entirely the same, but it covers enough to be really useful as a search term. This feature is of great value to beginners when it is integrated into an IDE well, as it helps them immediately focus on broken pieces of code rather than finding them by more indirect means. In hindsight, I wish I had focused more on getting plain Python and Jython to be more friendly in terms of "Edit and Continue": http://www.google.com/search?hl=en&q=edit+and+continue like by hacking the core C and Java code, and perhaps IDLE and PyDev, rather than focusing on using plain-vanilla Python to support prototypes, to add properties, or wrapping various widget libraries. After all, Smalltalk (Squeak or VisualWorks or several others) all allow "edit and continue" even though they are, like Python, mostly class based. In the end your (Kirby's) points have proven all too valid -- it's hard to get a new paradigm accepted, especially when it goes against the grain of how underlying libraries are written, all the books out there, and so on. The path of least resistance really would have been to add an "edit and continue" patch to C-Python (or Jython), perhaps centered around a PEP. It's not what I wanted -- since I wanted backward compatibility with older Python versions -- but in the end it I feel it would have been more effective, and likely a better payback for the person-months of effort that went into the PataPata experiment. Still, that's why people do experiments, to learn, and I learned. :-) Hopefully others can benefit from what I learned too. I still feel one thing PataPata got right (as a technical advance, inspired by Self) was showing how you could save and restore the state of a world of Python objects (and their behaviors) just by writing a plain text (and highly readable) Python program which reconstructed that world of objects (with that made slicker by using Ian Bicking's suggestions for meta class uspport). See my "postmortem" self-critique (for more information: "PataPata critique: the good, the bad, the ugly" http://patapata.sourceforge.net/critique.html I feel that's a good idea to keep in the back of your mind when thinking about OLPC applications and "view source". And it's so much nicer to do things that way than via pickle which produces an almost unreadable (and so effectively unmodifiable) output of a live set of Python objects. On "edit and continue", from Microsoft's web site: http://msdn2.microsoft.com/en-us/library/bcew296c.aspx "Edit and Continue is a time-saving feature that enables you to make changes to your source code while your program is in break mode. When you resume execution of the program by choosing an execution command like Continue or Step, Edit and Continue automatically applies the code changes with some limitations. This allows you to make changes to your code during a debugging session, instead of having to stop, recompile your entire program, and restart the debugging session." I pick Microsoft's definition even though they were late to the party (compared to Lisp and Smalltalk doing this decades ago) just to show how mainstream such technology has become. Isn't this omission just a glaring embarrassment for Python at this point? And no, module reloading doesn't cut it, as it entails building your own hooks into your application as well as thinking through a lot of implications of what that means -- definitely not something for Python newbies to consider doing. I myself add limited support for module reloading in a commercial application while developing and it makes a lot of difference when developing an application which has long load times or long processing times -- but it's not a trivial thing to manage, nor is it the same as "Edit and continue". So I think another attempt towards PataPata's ideals, but not emphasizing prototypes, and considering changes to the Python and Jython cores (C and Java), might be worth trying from a Python community point of view -- or even as Python3000 feature. Sadly, I'm now wrapped up in other things and can't devote even a few weeks (or more) to realizing this right now; plus I do think it is so central to the system it would take a while (for me) to get up to speed on C-Python internals and probably even longer to get such potentially radical changes accepted if submitted from someone as a first patch to C-Python. I know everyone tries to get Guido to add his or her own pet feature to Python, and so am I, :-) but look, if this was a fairly straightforward independent change like a bug fix or just a library add-on, anyone could do it; even though the code involved is small, this takes a true snake wrangler to get sure this is done right, and an overall commitment by the Python leadership to make sure it keeps working right as other changes are made to Python in the future. There are several issues, but the most important ones probably revolve around better support within the Python library for linking code to a place in a file (some parts of that are broken, like for one liner functions), within an IDE like IDLE building on that to allow fine-grained browsing of changes to just a single function in version control, and within the Python core dealing with issues with more easily allowing functions to be modified without producing new function pointers. Restartable exceptions would be ideal for this, but even without them, this could be doable -- just have the system break on creating certain specified exceptions before throwing them. I think it's a shame that Python can't more easily support this sort of "edit and continue" development; it already has most of the hooks. Just a few specific almost trivial things it lacks -- and I think it lacks them more from a conservative mindset than anything else -- an emphasis that restarting a program when you make a small change or hacking each one individually to reload modules is somehow still OK in the 21st century. :-) I'd simply hate to think this related comment by someone else was true: http://www.mail-archive.com/pypy-dev@codespeak.net/msg02446.html "you stand a better chance of doing this in pypy than the python developers do. they consider it "impossible", because it's too radical a rethink, because they have been trapped, by design, in c-code." Anyway, Smalltalk, Lisp, and other languages (including now Visual Basic and most languages from that vendor) have been doing this for decades. I hope OLPC can be the impetus to overcome this glaring omission with Python; otherwise, the OLPC group may find their entire approach to using Python instead of, say, Lisp or Smalltalk or Ruby or several other languages for the core was a mistake. Still, I'm sure at this point it would be much easier for the OLPC project to fix Python than to switch. Well, I guess OLPC could stick with Python as is, but the View Source key will always be a bit of a disappointment, as it won't be an "Edit and Continue" style "View the source you you can edit the running program like Squeak does and see your changes work immediately" sort-of key, but more a "View the source and that's it for now, maybe you can restart later" kind of key. This issue is painted by you (Kirby's) as "maturity" -- that kids are someday mature enough to use something like, say, Microsoft VisualStudio. I think that, while true in general that programmers need to learn to use a variety of tools as they mature, it is misleading in this particular situation -- considering how other systems like Lisp or Smalltalk have been doing "edit and continue" for decades -- allowing you to *easily* modify a running program and continue. The obvious issue IMHO is "immaturity" -- there is a piece of Python or its community culture which is lacking. And once Python gets that feature as part of its continuing maturation process, watch out, because after adding easy "Edit & Continue" internal support, and then using that support from an IDE like IDLE or PyDev, then Python will grow overnight to 10X the creature it is already! :-) Not to bash Python too much here; I use Python despite its glaring shortcoming in this area because it is so wonderful in many other important ways (licensing, libraries, readable syntax, helpful community, cross-platform support, namespaces, etc.) which in practice often outweigh this lack of "edit and continue". I mainly just want to see Python get even better -- especially considering the time and learning and code investment I have in it. Rather than agree with the comment above about being locked into a corner by using C, I could speculate there is too much of a C developer "culture" holding out there somehow (which is perfectly fine most of the time as Python is written in C, and C can be a great language if used well. It's just that thinking in C culture terms (C culture --> edit, compile, start, run, crash, debug, stop, edit, compile, restart, repeat) just hurts a lot in this one particular area (where Smalltalk culture -> start, crash, debug, crash, debug, crash, etc. when you edit code in the debugger). Somehow I feel if Guido could just see, say, VisualWorks (or ideally VisualWorks+ENVY) in action for an hour by an expert (not Squeak, which has too many other ideas overloaded on it), he'd see what was really possible and desirable with dynamic objects. Gee, maybe I should Google that? Hmmm... Well what do you know! :-) Here is a six minute example: "A nice demo of Smalltalk development in the debugger by David Buck." http://motionobj.com/article/demo-of-smalltalk-visualworks Links to: http://www.simberon.com/smalltalkdemo.avi The video above shows the cycle of "Debug, Crash, Debug, Crash, etc." while incrementally developing a Solitaire-style card game, except a crash is really just an exception thrown and launches the debugger, where the problem is fixed, and you're able to continue. How can anyone watch this six-minute video of VisualWorks and not want Python (and OLPC) to be able to do the same thing and just as easily? (Note the debugger, browser, inspector, and selected code evaluation all being used together -- that's really what it takes to do good incremental coding -- but it is also almost all it takes. :-) Still, I do think there is a fundamental issue when thinking about "view source" key in interaction with a GUI, since many GUIs are generated by a function, so it is always potentially unclear whether you are modifying the instance of the GUI you are looking at or are modifying the method which generates that GUI. I think that issue is approachable, but it does add layers of conceptual complexity. Prototypes in PataPata resolved some of that conceptual complexity (as live prototypes could defined the interface), but at a cost of adding complexity to Python in terms of support libraries and using Python in a non-standard way which made problems harder to debug. So, by moving away from prototypes and back to classes (including using the PythonCard-like approach for building GUIs hung off a central class holding all the behavior of individual widgets instead of a PataPata approach where widgets had significant behavior stuck in them), one needs to rethink how to handle that approach. One generic thought for the far future I had on that was that each pixel on the screen should be tried to a stack trace as the code was when that pixel was written, however, even that might miss the link to where the objects performing the behaviors in question were initialized. In that sense, I think Smalltalk's approach of supplying a code browser, inspector, and debugger, and with a developer typically using all three at the same time as a team to figure out where objects of interest were initialized, is really the way to go (and also evaluating snippets of code from any handy text pane). Still, I think you (Kirby) are right in general on the issue of a multiplicity of editors for various situations. And I think that is a good thing for people to eventually learn. However, specifically, the reality is that most behavior of interest (in terms of changing) on something like the OLPC (especially to novices) is likely to be coded in Python. This also shows the value of Jython and Java for Python development on other systems, as it would be easy to make a system that lets you tinker with both levels (Python=slow but smart code, Java=fast but dumb code) since Java has better reflective capabilities than C; still C could be compiled in debug mode to make such information more available, and even run-time modifiable by invoking the compiler and some sort of linker for modified functions. Though also, in the VisualWorks case, most Smalltalk code typically runs about 6X slower than the speed of C or so, which is generally good enough (rather than Python sometimes approaching 100X slower for loops and motivating more C coding, and thus better C tool integration). And your (Kirby's) emphasis on multiple editors suggest this idea: one way to approach the "View Source" key on the OLPC project that might be to treat "view source" as a "meta key" and perhaps press other keys at the same time to get at various aspects of the running program -- P=process list, G=GUI widgets, M=underlying modules, I=initialization code, L=command line, T=current thread, S=source code manager, B=breakpoints (including on certain E=exceptions), B=loaded binary modules (and associated source and editors like if in C), and so on. --Paul Fernhout kirby urner wrote:
Here's a floating email fragment
Date: Thu, 28 Jun 2007 20:54:09 -0400 From: "Walter Bender" <walter.bender@...> Subject: OLPC software: our first release and beyond. To: "OLPC Devel" <devel@...>, sugar@...
One important feature where we have made little progress is the View Source key. We decided to build the user experience?and the core Activity base?in Python, in large part, to facilitate the children's direct access to modifiable code. We are moving to the latest stable version of Python as part of our Fedora Core 7 migration; however, we have yet to put enough resources into building a suitable development environment for children. This remains an important goal, but not one we can reasonably meet for our first release. The incorporation of the context-sensitive spirit of "view source" into all Activities is another area where we lag. Bolstering these efforts is second only to stabilizing the current system. We look forward to the possibility of Guido van Rossum, Python's creator, leading these efforts in the fall.
Always interesting to see where some people are hoping to invest some of our BDFL's precious clock cycles...
Stepping back for a moment, one has to realize this is a different shop talk, as we typically look at source code as text files, modifiable yes, but not while running as binaries, which are different files entirely, derived from said text files by various processes.
Some of this text is expressed in pure Python, but a lot of what *runs* under Python in modular form is actually written in something else, as is Python itself. A whole different text editor might be more appropriate in that case, or at least different plug-ins (the Eclipse model).
The trend is to target a shared VM and play freely with other language communities, swapping whatever works in inter-pluggable arrangements.
One way around this shortcoming of Sugar's is to suggest that once kids are mature enough to eyeball the kind of convoluted source code required to keep the likes of Sugar alive, it might be time to transition to a more standard model laptop that doesn't push the bleeding edge so hard.
Adults need to focus on stable developer environments like Wing's or whatever (or like IDLE for starters). So at your bat mitzvah or other ceremony, you maybe trade in your much loved OLPC XO (and your teddy bear), and move up to something more expensive, with a more traditional source code treatment (anyone ready for C yet?).
Or just start using more of those legacy TuxLab desktops, running a variety of distros (Ubuntu Studio might draw some kids, more because they want to study graphical arts than they want to do lexical programming).
Basically, I'm somewhat Kusasian in outlook: go with immersive up to a point, then take a break and study some classic combos e.g. emacs + bash + gcc. Drop all the way down to machine language even, coming up through MMIX enroute to Python Part Deux (or "snake guts revisited").[1] At some Coming of Age level, we no longer want to shield kids from the messy reality of *no* single governing paradigm at all levels and the old immersive environment turns out to be one fish tank among many.
In retrospect, those early experiences with special namespaces for children will remain a comfort (we hope), and some will elect to professionally teach them to a next generation, and/or invent new ones (already the case, so a next iteration).
Kirby
[1] http://worldgame.blogspot.com/2007/06/techie-chatter.html