reload()'ing modules and how it affects references

[Thomas Wouters]

On 26 Jun 2000 21:43:52 +0200, Igor V. Rafienko <igorr at ifi.uio.no> wrote:

>[the article is a little bit long -- sorry]

>I'm trying to understand how reload() works. "Learning Python" and
>Python's documentation warn against the dangers of combining 
>from ... import ... and reload(), so that is fine, I use 
>"import modulename".

The problem with 'from module import something' and 'reload()' is not the
magic word 'from': It's the fact that 'from module import something' is a
combination of 'import module' and 'something = module.something'.

The best way to look at this is probably by thinking of the reload()ed
module as a *different* module, which just happens to have the same name.

Lets see:

>import sys
>import fooclass
>
>ml = [ fooclass.Foo, fooclass.Foo() ]
>while 1:
>    print ml[0].Y, ";", ml[1].Y
>    next = sys.stdin.readline()
>    if not next: break
>    else:
>        exec next
>    print ml[0].Y, ";", ml[1].Y

># fooclass.Foo.Y is printed, runtime1.py import's fooclass
>$ ./runtime1.py 
>5 ; 5

You start runtime.py, which loads fooclass, and stores a reference to
fooclass.Foo *and* an instance of fooclass.Foo in a list. The instance of
fooclass.Foo keeps its own reference to fooclass.Foo, by the way, in the
.__class__ attribute.

>			# here, I've changed fooclass.Foo.Y to 6 in emacs
>reload( fooclass )
>5 ; 5
>5 ; 5

Right. You reload the module, but the list containing the class and the
instance point to the *old* value. You don't point to a name, in Python, you
point to an object, and the object hasn't changed -- the old name simply
points to a new object.

>			# now re-assign the element explicitly
>ml[0] = fooclass.Foo
>6 ; 5
>6 ; 5
>			# and the instance
>ml[1] = fooclass.Foo()
>6 ; 6
>6 ; 6

These work, of course, because you point the variables to the new objects by
hand.

>(Apparently it does not matter whether the class reference/class
>instance reference is stored in a variable or a list)

Nope. Nor does it matter wether it's done in a tuple, a dictionary, a
builtin type that holds references, an extention type someone made that
holds references, or a user-defined class (which can hold references as
well.)

A variable is simply a container for a single reference. 'something =
fooclass.something' simple takes the reference that is stored in
'fooclass.something', and assigns it to 'something'. It doesn't look at the
value, it doesn't keep tabs on 'fooclass.something' to see if it's changed.
A list is very similar, except that it can be a container for an arbitrary
number of references, and is itself stored in a variable.

>Now, for the questions:

>1) What is the most elegant way of assigning all references to a
>certain class to a new definition of this class after a reload? (I can
>keep a dictionary/list of all the references, and assign them
>explicitly right after the reload() call, but it just looks too ugly).

There isn't really one. What helps for functions is if you do the 'import' +
assignment inside functions, so that they always have the newest version.
This doesn't help for classes defined in fooclass, however. You can change
an instance's __class__ attribute to point to a new parent, but you'll have to
keep a list of the instances first ;-P

reload() isn't really intended to change the behaviour of whole programs in
mid-run. You can do it, but only if you consider all information in the
relevant module as 'volatile' -- you have to do a full lookup each time. And
that includes not using classes defined in that module.

>I suppose that it is impossible to make an instance of a class
>suddenly "become" an instance of the new definition of the same class,
>so that is fine.

Nothing is impossible in Python ! :-)

>>> class A:
...     pass
>>> class B:
...     pass
>>> a = A()
>>> a
<__main__.A instance at 80c9790>
>>> a.__class__
<class __main__.A at 80c97f0>
>>> a.__class__ = B
>>> a
<__main__.B instance at 80c9790>

(You knew you could do your little while 1: readline program above in the
interpreter instead, right ? Without having to code a small interpreter
using exec, and with the ability to peek at more things ? :)

The problem with above code is that it skips B's __init__ method, so it
might not be properly initialized.

>2) When I call a method in class Foo that looks something like this:

>def __repr__( self ):
>    print <In Foo: Foo.Y ==", Foo.Y, "; self.Y == ", self.Y, ">"

>after a reload the changes to Foo.Y affect Foo.Y but _not_ self.Y.
>That is, the output from the test similar to the one above looks like
>this (the function I call is __repr__):

>$ ./runtime1.py 
>6 ; 6 ; <Foo instance: self.x == 3, Foo.Y == 6, self.Y == 6>
>reload( fooclass )    
>	# now Foo.Y == 5 in the module file
>6 ; 6 ; <Foo instance: self.x == 3, Foo.Y == 5, self.Y == 6>
>6 ; 6 ; <Foo instance: self.x == 3, Foo.Y == 5, self.Y == 6>

>Why does it happen -- I thought that Foo.Y and self.Y were supposed to
>refer to the same object from within a class method? Am I missing
>something essential?

Yes :) Think of it in references. 'self.Y' searches itself for
a Y attribute, and if it can't find it, searches its own __class__
attribute, which points to a class object, for the attribute. But as we've
seen above, self.__class__ points to the *old* Foo.

Foo.Y, however, first searches for Foo. It isn't defined in the __repr__
function, so it's searched for in the global namespace, the fooclass module,
where Foo is defined. But this is the *new* Foo class, so it contains the
new value.

Think of reload creating a new module with stupidly the same name (silly
reload function, why does it do that ? Confusion galore !) and all will be
well ;-)

reload()'s-a-bitch-and-then-you-marry-one-ly y'rs,
	Thomas.