Class destructor -- strange behaviour

[Gabriel Genellina]

En Thu, 06 Dec 2007 18:51:08 -0300, Spes <Vlastimil.Holer at gmail.com>  
escribió:

> I have this simple code:
> | #!/usr/bin/python
> | import codecs
> | import re
> | from copy import deepcopy
> |
> | class MyClass(object):
> |   def __del__(self):
> |     deepcopy(1)
> |
> | x=MyClass()
>
> but I get an error:
> | Exception exceptions.TypeError: "'NoneType' object is not callable"
> in <bound method MyClass.__del__ of <__main__.MyClass object at
> 0x6fcf0>> ignored

Good question!
First: what does the message mean? Somewhere inside __del__, you are  
trying to call an object, and that object is None. The only possible cause  
is deepcopy being None, and you can confirm that easily: adding a 'print  
deepcopy' just above the call, shows that deepcopy is actually None inside  
__del__.

This is related to the Python finalization process. At some stage, Python  
clears the __main__ module's namespace: not by removing the names, but  
assigning None to them, and finally removing __main__ from sys.modules.  
(The same thing is done, in turn, for all other modules, ending with sys  
and __builtin__). That means that you can't trust any globals inside a  
destructor (see the second big warning at  
http://docs.python.org/ref/customization.html).
You can avoid this problem (partially) storing a reference to deepcopy  
into the *local* namespace:

	def __del__(self,
	               deepcopy=deepcopy):
	  ...
(this works as long as the deepcopy function itself does not reference  
other globals that might not be still available)

> The problem disappears if I do anything of this:
> 1. change
>      - from copy import deepcopy
>      + import copy
>     and call directly copy.deepcopy(1)
>
> or
> 2. don't store object to variable `x'
>
> or
> 3. don't import module `re'

Back to the finalization process, the names in the __main__ module  
namespace are set to None IN THE ORDER IN WHICH THEY ARE ENCOUNTERED (in  
fact, this is done in two stages: first names starting with '_', then all  
remaining names, but excluding '__builtins__'). So, if "deepcopy" (as a  
dictionary key) comes before "x" (the variable name) when you iterate the  
module's dict, deepcopy will be set to None before x, and when x.__del__  
is called it will fail.

Now try your examples again and print globals.keys() for each variant, and  
notice the name ordering. If you can arrange things so 'x' (or whatever  
variable name you choose) comes before than 'deepcopy' in globals.keys(),  
you won't get any error.
Note that, for example, types are irrelevant: you can replace 'import re'  
with 're=0' and get the same results.

> The first solution is OK, but I would like to know why it behaves so
> strange. We have tested on:

I hope my explanation is understandable. If you want to know the gory  
details, see Py_Finalize in pythonrun.c, PyImport_Cleanup in import.c and  
_PyModule_Clear in moduleobject.c

> - Mac OS X Tiger for PPC
> Python 2.3.5 (#1, Mar 20 2005, 20:38:20)
> [GCC 3.3 20030304 (Apple Computer, Inc. build 1809)] on darwin
>
> - Linux 64bit and 32bit
> Python 2.4.4 (#1, Oct 30 2007, 14:31:50)
> [GCC 4.1.2 (Gentoo 4.1.2 p1.0.2)] on linux2
> Python 2.5 (r25:51908, Jan 12 2007, 13:57:15)
> [GCC 4.0.2 20051125 (Red Hat 4.0.2-8)] on linux2

Mmm, you got the same results on 64 bit too? I would have expected that,  
having a 64 bits hash value, key ordering inside a dictionary would be  
different. Perhaps it's just a coincidence.

-- 
Gabriel Genellina