low-end persistence strategies?
John Lenton
john at grulic.org.ar
Wed Feb 16 23:58:13 EST 2005
On Tue, Feb 15, 2005 at 06:57:47PM -0800, Paul Rubin wrote:
> I've started a few threads before on object persistence in medium to
> high end server apps. This one is about low end apps, for example, a
> simple cgi on a personal web site that might get a dozen hits a day.
> The idea is you just want to keep a few pieces of data around that the
> cgi can update.
>
> Immediately, typical strategies like using a MySQL database become too
> big a pain. Any kind of compiled and installed 3rd party module (e.g.
> Metakit) is also too big a pain. But there still has to be some kind
> of concurrency strategy, even if it's something like crude file
> locking, or else two people running the cgi simultaneously can wipe
> out the data store. But you don't want crashing the app to leave a
> lock around if you can help it.
>
> Anyway, something like dbm or shelve coupled with flock-style file
> locking and a version of dbmopen that automatically retries after 1
> second if the file is locked would do the job nicely, plus there could
> be a cleanup mechanism for detecting stale locks.
>
> Is there a standard approach to something like that, or should I just
> code it the obvious way?
one easy way would be something along the lines of
from ConfigParser import ConfigParser
from fcntl import flock, LOCK_SH, LOCK_EX, LOCK_UN
class LockedParser(ConfigParser):
def _read(self, fp, fpname):
flock(fp, LOCK_SH) # block until can read
try:
rv = super(LockedParser, self)._read(fp, fpname)
finally:
flock(fp, LOCK_UN)
return rv
def write(self, fp):
flock(fp, LOCK_EX) # block until can write
try:
rv = super(LockedParser, self).write(fp)
finally:
flock(fp, LOCK_UN)
return rv
although you could do the same kind of stuff with csv, or even
Pickle. Of course this doesn't work if what you're wanting to
implement is a hit counter, but that is much easier: just grab a
LOCK_EX, read in, write out, LOCK_UN. If you care about (not)
overwriting changes, but fear you'll hold the lock for too long with
the simple 'grab the lock and run' approach, you could save a version
of the original file and compare before writing out. Complexity grows
a lot, and you suddenly would be better off using pybsddb or somesuch.
Of course I'm probably overlooking something, because it really can't
be this easy, can it?
--
John Lenton (john at grulic.org.ar) -- Random fortune:
BOFH excuse #44:
bank holiday - system operating credits not recharged
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20050217/e5a23de2/attachment.sig>
More information about the Python-list
mailing list