[Python-bugs-list] [ python-Bugs-702775 ] dumbdbm __del__ bug

[SourceForge.net]

Bugs item #702775, was opened at 2003-03-13 01:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=702775&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Jane Austine (janeaustine50)
Assigned to: Nobody/Anonymous (nobody)
Summary: dumbdbm __del__ bug

Initial Comment:
I used shelve.py and it falls back on dumbdbm when no
possible alternatives are found on the system.

I found this error, which is recurrent and deterministic:

Exception exceptions.AttributeError: "'NoneType' object 
has no
attribute 'error'" in <bound method _Database.__del__ of
<dumbdbm._Database instance at 0x820c71c>> ignored

The problem seems to reside in the __del__ of 
dumbdbm._Database:

class _Database:
...
    def __del__(self):
        if self._index is not None:
            self._commit()
...
    def _commit(self):
        try: _os.unlink(self._bakfile)
        except _os.error: pass
        try: _os.rename(self._dirfile, self._bakfile)
        except _os.error: pass
        f = _open(self._dirfile, 'w', self._mode)
        for key, (pos, siz) in self._index.items():
            f.write("%s, (%s, %s)\n" % (`key`, `pos`, `siz`))
        f.close()

My investigation showed that the error was from 
_commit. When
it was called, _os or _open was both None. And the 
exception
catch didn't work quite safely cause its in the "except" 
clause.

The reason I suspect is when the time that 
_Database.__del__ was
called the os module(which is imported as _os) is 
already removed out.

I changed the code as:

    def _commit(self):
        global _os
        if _os is None:
            import os as _os
            import __builtin__
            _open = __builtin__.open
        try: _os.unlink(self._bakfile)
        except _os.error: pass
        try: _os.rename(self._dirfile, self._bakfile)
        except _os.error: pass
        ......

Now it works without any problems, AFAIK.



----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-03-25 23:09

Message:
Logged In: YES 
user_id=31435

Neal, this is actually a common problem in __del__ 
methods, and the OP's analysis is on target.  When Python 
is shutting down, it tries to tear down module dicts in a 
safe-as-possible order, but modules are full of reference 
cycles and there is no *wholly* safe order.

In general, a __del__ method should never reference 
globals because of this.  The usual solution can be seen in 
tempfile.py:  store the global objects __del__ will need as 
class attributes when the class is created, and refer to 
these bindings in __del__ via self.attrname 
(ClassName.attrname is also no good, because it refers to 
the global ClassName!  that may also become None).

Reimporting a module instead may not be effective (if it's in 
a partially torn-doiwn state but its name is still a key in 
sys.modules, importing again will just retrieve the partially 
torn-down module object).  The class-attr trick is robust.

----------------------------------------------------------------------

Comment By: Jane Austine (janeaustine50)
Date: 2003-03-25 23:08

Message:
Logged In: YES 
user_id=732903

Tested on linux and windows xp with Python2.2.2 and 
Python2.3a2. 

----------------------------------------------------------------------

Comment By: Jane Austine (janeaustine50)
Date: 2003-03-25 23:04

Message:
Logged In: YES 
user_id=732903

Run the main.py in Python 2.3+ putting the two files in the 
same place.

For Python 2.2+, put the two files in the same package and 
make a new file that imports main.py and run it as follows:

=======
from <packagename> import main
=======

----------------------------------------------------------------------

Comment By: Jane Austine (janeaustine50)
Date: 2003-03-25 22:55

Message:
Logged In: YES 
user_id=732903

A test case that triggers this problem:

#foobar.py
def open(filename, flag='c'):
    import dumbdbm
    return dumbdbm.open(filename,flag)

c=open('test.dbm')

#main.py
import foobar

$ python main.py
>>> 
Exception exceptions.TypeError: "'NoneType' object is not 
callable" in <bound method _Database.__del__ of 
<dumbdbm._Database instance at 0x401e482c>> ignored

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-03-18 20:06

Message:
Logged In: YES 
user_id=33168

Can you provide a test case which triggers this problem?  I
can't see how _os becomes None.  Also I would like to add a
test.  Thanks.

----------------------------------------------------------------------

Comment By: June Kim (juneaftn)
Date: 2003-03-13 12:02

Message:
Logged In: YES 
user_id=116941

see the thread at http://groups.google.com/groups?
selm=ba1e306f.0303111337.72a696c7%
40posting.google.com , esp. by my name.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=702775&group_id=5470