[Spambayes] Concurrent DB access leads to corruption? (FAQ question)

[wsy]

   From: Skip Montanaro <skip at pobox.com>

       Jim>        1. Accessing the database files from more than one process
       Jim>           concurrently.

       Jim> Is this information current?

   Yes, I believe it still is.  Most SpamBayes usage scenarios involve just one
   of the applications, so there's been no crying need to add locking so they
   can operate concurrently.  I think it would be fairly difficult to implement
   anyway.  Some apps, like sb_filter, only keep the db open briefly.  Others,
   like sb_imapfilter and sb_bnfilter (via it's background server) want to keep
   the database open for long periods.  An application waiting on one of these
   applications to release the lock would probably need some way to signal the
   long-running app to close the database.

       Jim> The cron task could fire at the same time as procmail. Does sb use
       Jim> internal locks to prevent this situation? Is there otherwise a way
       Jim> to avoid corruption with unavoidable concurrent access?

   Not that I'm aware of.  Feel free to implement something and toss it back
   over the wall. ;-)

Toss it over my wall too, please!  :-)

The problem seems to be that if you have an NFS-mounted directory 
containing your files of interest, there is *no* gauranteed way 
to lock it without using an "out of band" communications system that
gaurantees sequential serialized delivery of messages with confirmation.
TCP/IP is one such, but NFS semantics are _not_.

So... unless you want to add a TCP-accessed lock server, there doesn't
seem to be a way to provide gauranteed (not just unlikely,
*gauranteed*) locking, even among processes that *want* to cooperate.

If I am wrong in this, *please* tell me how to do it.  I want to know.

(and no, lockfile(1) is merely NFS-resistant, not NFS-proof; it will fail
to prevent corruption under certain loadings and/or client/server mixes).

      -Bill Yerazunis