[spambayes-dev] A new and altogether different bsddb breakage

[Kenny Pitt]

Barry Warsaw wrote:
> On Wed, 2003-12-17 at 18:25, Richie Hindle wrote:
>>  o Since BDB uses bsddb in transaction mode rather than single-file
>>    mode, we can say goodbye to those nasty little DBRunRecovery
>> errors.  Yay! 
> 
> That's the hope, anyway. :)

Unfortunately I don't think we can say that with any confidence until we
know why they are occurring in the first place.  The following comments
apply to using bsddb directly.  I'm not familiar with ZODB, and don't
know if they are already handling all of these issues.

The BerkeleyDB docs say this:

"""
Errors can occur in the Berkeley DB library where the only solution is
to shut down the application and run recovery (for example, if Berkeley
DB is unable to allocate heap memory). When a fatal error occurs in
Berkeley DB, methods will throw a DbRunRecoveryException, at which point
all subsequent database calls will also fail in the same way. When this
occurs, recovery should be performed.
""" (http://www.sleepycat.com/docs/api_cxx/runrec_class.html)

This seems to indicate that this problem can be caused by more than just
threading problems.  It also says this:

"""
When building transactionally protected applications, there are some
special issues that must be considered. The most important one is that
if any thread of control exits for any reason while holding Berkeley DB
resources, recovery must be performed...
""" (http://www.sleepycat.com/docs/ref/transapp/app.html)

This seems very clear that using full transactional mode does not
protect you from DbRunRecovery errors.

I wonder if the only real solution is to run recovery when opening the
database.  This should be easy for the Outlook add-in, sb_server, etc.
where a single, long-running process performs all database access (just
specify the DB_RECOVER flag when opening the environment).  Running
recovery requires that only one thread in one process has access to the
database environment until recovery is complete.  This might be harder
to accomplish for apps such as sb_filter that could be run as multiple
simultaneous processes.

-- 
Kenny Pitt