[Python-3000] Is reload() feasible?

[Jack Diederich]

On Thu, Apr 13, 2006 at 01:39:43PM -0500, Ian Bicking wrote:
> In another thread ("Module Initialization and finalization") reload() 
> has come up, and some suggestions to improve the reloadability of modules.
> 
> I would like to ask: is reloading feasible at all?  Right now reload() 
> works poorly.
[snip]
> To focus energy I think we should give up on reloading.  At the same 
> time, long running processes that require manual restarts just *feel* 
> really uncomfortable.  It's like a compile step.  So the underlying 
> problem shouldn't be ignored.

+1 reload() is OK for the interactive shell but worthless in real
situations (e.g. anything that uses classes or custom exceptions).

> I would propose that light, disposable interpreters are a better way of 
> handling the problem.  When a file changes, you throw away the 
> interpreter and start over.  Several projects are doing this now, but 
> making it more standardized would be very helpful to improving the 
> usability of the technique.

A standard way to hide objects across interpreter restarts would be
great.  For anything with an open socket if you could hide it from
GC, serialize your current state (if the app is stateful) and then
start fresh with a new interpreter, and start using the socket again.
Will an object that has been PyObject_GC_UnTrack() survive across
a Py_Finalize()?  If so a hidey-hole for objects would be fairly
easy to implement.

Stateful applications would still have to serialize most of their
state and reload it across interpreters.  Trying to hide instances
of python-defined objects in the hidey-hole has all the same problems
as reload().

> For interactive environments some of the same issues apply, though the 
> resolution is more difficult.  Can you safely re-evaluate the entire 
> history and get to the same state that you started at?  You can imagine 
> using restart() at the interactive prompt, and having the process die, 
> restart, run the entire history (showing output?) and leaving you with a 
> potentially equivalent environment.  Replaying history also might fail, 
> or have side effects that cannot or should not be repeated.  So, a 
> restart strategy opens up lots of questions; but I think abandoning 
> reload() would better bring those questions into focus.

Some apps might not have internal state that easily lends itself
to being serialized and deserialized but those same apps don't work
with reload either.  Mostly stateless apps like web servers would
have an easy time saving and restoring what little state they have.

The name "hidey-hole" could use a bit of work, though.

-Jack