[spambayes-dev] DBRunRecoveryError

[Tim Peters]

[Richie Hindle]
> Here's a script that will either crash Python 2.3c1 ("illegal
> operation") or give a DBRunRecoveryError - no need for your mirror
> dict here!  This is on Win98, untested on other OSs.

I also got a DBRunRecoveryError on Win2K.

> I suspect this is a red herring from Spambayes' point of view because
> I don't believe the message DB is ever shared between threads,

Hard to say -- there are so many clients.  ZODB was designed to be
thread-safe without any special diddling, but most other dbs aren't.

> but here it is anyway:
>
> ----------------------------------------------------------------------
>
> import os, random, bsddb, thread
>
> DB_FILENAME = "hammer_bsddb.db"
>
> def hammer(db):
>     for i in range(1000000):

Use xrange here instead to save a ton of RAM.

>         rnd = random.random()
>         if rnd < 0.33:
>             # Add/update a value.
>             value = "This is a test"[:int(random.random() * 10)]
>             db[str(int(random.random() * 100000))] = value
>             elif rnd < 0.66: # Read a value.
>             try:
>                 x = db[str(int(random.random() * 100000))]
>                 except KeyError: pass
>         else:
>             # Delete a value
>             try:
>                 del db[str(int(random.random() * 100000))]
>                 except KeyError: pass
>
> def main():
>     try: os.remove(DB_FILENAME)
>     except OSError: pass
>     db = bsddb.hashopen(DB_FILENAME, "c")


For Barry's info, I still saw

  File "C:\Code\python\lib\bsddb\__init__.py", line 90, in __setitem__
    self.db[key] = value
bsddb._db.DBRunRecoveryError: (-30982, 'DB_RUNRECOVERY: Fatal error, run
database recovery -- fatal region error detected; run recovery')

after changing the line above to

    db = bsddb.hashopen(DB_FILENAME, "c", bsddb.db.DB_THREAD)
                                          ^^^^^^^^^^^^^^^^^^


>     thread.start_new_thread(hammer, (db,))
>     thread.start_new_thread(hammer, (db,))
>     thread.start_new_thread(hammer, (db,))
>     thread.start_new_thread(hammer, (db,))
>     hammer(db)
>
> if __name__ == '__main__':
>     main()
>
> ----------------------------------------------------------------------
>
> The docs for bsddb say nothing about threadsafety, but ISTR people on
> this list claiming that it is threadsafe.  Like I say, I don't think
> threading is Spambayes' problem, but I'm ready to be corrected...

Well, you never know what the hell Outlook will do to you <0.7 wink>.