[Edu-sig] Reloading code (was Re: OLPC: first thoughts)

[Paul D. Fernhout]

Guido van Rossum wrote:
>>In some ways, xreload's is better by including a general "Provide a hook
>>for updating" concept. However xreload just ignores what to do with top
>>level objects like lists or dicts -- xreload apparently throws away the
>>updates to them.
> 
> 
> No it just replaces the whole object. I was just punting on this; what
> are the required semantics?

You're right; I read this wrong.

The point about required semantics for global objects when reloading their 
modules is that there are none. As you say, one might imagine different 
situations requiring different effects when you reload a module. Perhaps 
you could make a pragma directive, or define a reload hook like you set 
up, but even then the need might vary depending on how you are developing 
(i.e. is the point of the reload because you changed a method or because 
you changed that specific global data structure?). The point is that 
reloading an entire module forces you to think about this -- reloading 
just a method lets you ignore the problem.

> There just is no way to handle *all* possibilities. How would you
> handle renamings?

Selectively in the IDE? :-)

> The GUI part isn't supposed to be included. There's also the issue of
> what to do with dependencies; if x imports y, and y is reloaded, x
> probably needs to be reloaded too...

Yes -- another reason not to want to reload an entire module if it can be 
avoided. Although in practice, this seems a bigger problem when doing a 
regular reload rather than when doing an xreload type thing (since other 
modules which are handing onto the classes get the new behavior).

>>Still, elegant as it is (including the reload hook), even xreload does not
>>handle all the important cases -- just (forgive me :-) the *easy* ones.
>>
>>When Guido supplies an xrestart.py :-) python 2.5 code module that lets
>>you restart exceptions after having modified one of the functions in the
>>stack trace, then I will admit it is something I have never seen before in
>>Python and be suitably impressed. Is it even possible without modifying
>>the VM? :-)
> 
> 
> It is impossible because we don't know whether the exception has a
> handler (an except block that traps it) until we've bubbled all the
> way up.

OK, impossible as things are now. Why not keep the stack of internal 
objects (exception handlers?) around somehow so that after you have 
bubbled up, you can present that stack of objects to the debugger?
The point is not to go into the debugger on every exception. The point is 
that when you are debugging you can make changes -- either while stepping, 
or when an otherwise unhanded exception occurs. Essentially, think of it 
as a having default exception handler that throws you into the debugger at 
the point the exception is created (but before any related handling is 
done, like in finally clauses). I don't know the CPython internals on 
exception handling, so I do not know what would be easy to do. Perhaps you 
can not look up the stack of exception handlers to see if anyone would 
catch it without processing each handler's finally clause first? And even 
if you can, you would still need to hold that entire exception processing 
stack around so that if someone closes the debugger window (or entire 
application) without restarting the exception those finally clauses get 
done. It may require adding an entire level of abstraction to exception 
handling if it is impossible now?

Also, in the case of Smalltalk, one can always add a "Self halt" that 
throws an exception for sure which almost certainly reaches the debugger. 
It is common to create a method function with just "self halt" and when 
that is called, then fill out the method in the debugger.

Coding in the debugger is an entirely different style of software 
development. Sometimes it is very nice.
   http://www.google.com/search?hl=en&q=smalltalk+%22coding+in+the+debugger%22

Again, for small programs this may not matter, but when writing, say, a 
simulation which may run for hours, having to restart when you get an 
unhanded exception (like from a minor typo) can really harm productivity.

>>One impressive thing about the Python design which I liked from all this
>>was how Python separates the notion of execution code from a pointer
>>reference.
> 
> 
> Can you clarify this for someone who is not familiar with Smalltalk?
> (Your habit of explaining every idea by comparing it to how that idea
> was implemented in Smalltalk doesn't help the audience understand you
> -- you're preaching to the choir when you do that, and the rest of the
> congregation is lost.)

Smalltalk has a separation too of behavior (a method) from invocation 
using a "selector" or name of a method by sending a message -- I was just 
thinking how much better this was, than, say C, where a function's actual 
code location is tightly tied to the pointer to the function. So you can't 
take a C function pointer and just assign new code to it.

> 
>>That is what makes all these reloading tricks possible. And my
>>hat goes off to Guido for having included the extra level of indirection
>>which makes this feasible. I can hope that generality and late bindingness
>>might also make possible restarting an exception without VM changes.
> 
> 
> Why do you care about avoidung VM changes? The VM changes
> (incrementally) at each minor Python release.

Just so everyone (especially *me* :-) can use start using it right now, 
including in older versions of Python (like 2.4 etc.).

Of course, I would have to break that bad habit I've gotten into of 
restarting the program every time I make a change -- and it's hard to 
break that habit, even when I use a supplemental tool that lets me reload 
Jython modules selectively -- I keep thinking -- I did not have to 
restart. :-)

Still, even with all this, if you are making a GUI and modify the function 
that defines the window, you generally still need to close and open the 
window again. So there remain limits (unless you move to GUIs defined 
interactively like Morphic or PataPata). Just talking about getting most 
of the benefit.

--Paul Fernhout