shelf-like list?

[Chris Rebert]

On Sat, Aug 14, 2010 at 5:13 PM, kj <no.email at please.post> wrote:
> In <af7fdb85-8c87-434e-94f3-18d8729bf9be at l25g2000prn.googlegroups.com> Raymond Hettinger <python at rcn.com> writes:
>>On Aug 12, 1:37=A0pm, Thomas Jollans <tho... at jollybox.de> wrote:
>>> On Tuesday 10 August 2010, it occurred to kj to exclaim:
>>>
>>> > I'm looking for a module that implements "persistent lists": objects
>>> > that behave like lists except that all their elements are stored
>>> > on disk. =A0IOW, the equivalent of "shelves", but for lists rather
>>> > than a dictionaries.
>> . . .
>>> You could simply use pickle to save the data every once in a while.
>
>>That is a very reasonable solution.
>
> Sorry I don't follow.  Some sample code would be helpful.

I would assume something along the lines of (untested):

from pickle import dump

MOD_THRESHOLD = 42

class PersistentList(list):
    def __init__(self, filepath):
        self.filepath = filepath
        self._mod_count = 0

    def save(self):
        with open(self.filepath, 'w') as f:
            dump(self, f)
        self._mod_count = 0

    def append(self, *args, **kwds):
        super(PersistentList, self).append(*args, **kwds)
        self._mod_count += 1
        if self._mod_count >= MOD_THRESHOLD:
        # obviously time-since-last-dump or other
        #   more sophisticated metrics might be used instead
            self.save()
# define similar wrappers for list's other mutator methods:
__delitem__, __iadd__, __imul__, __setitem__, extend, insert, pop,
remove, etc.
# using decorators should help decrease code duplication

Cheers,
Chris
--
http://blog.rebertia.com