Special logging module needed

[Dan Stromberg]

On Wed, Mar 23, 2011 at 7:37 AM, Laszlo Nagy <gandalf at shopzeus.com> wrote:

>  I was also thinking about storing data in a gdbm database. One file for
> each month storing at most 100 log messages for every key value. Then one
> file for each day in the current month, storing one message for each key
> value. Incremental backup would be easy, and reading back old messages would
> be fast enough (just need to do a few hash lookups). However, implementing a
> high availability service around this is not that easy.
>

I think a slight variation of this sounds like a good bet for you.  But when
you "open" a database, create a temporary copy, and when you close the
database, rename it back to its original name.  Then your backups should be
able to easily get a self-consistent (if not up to the millisecond)
snapshot.

Or did you have some other problem in mind for the gdbm version?

BTW, avoid huge directories of course, especially if you don't have hashed
or btree directories.  One way is to come up with a longish hash key (sha?),
and use a trie-like structure in the filesystem on fibonnaci-length chunks
of the hash keys becoming directories and subdirectories.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110323/673073ae/attachment-0001.html>