shelf-like list?

kj at
Sun Aug 15 12:58:14 CEST 2010

In <mailman.2125.1281849995.1673.python-list at> Chris Rebert <clp2 at> writes:

>On Sat, Aug 14, 2010 at 5:13 PM, kj < at> wrote:
>> In <af7fdb85-8c87-434e-94f3-18d8729bf9be at> Ra=
>ymond Hettinger <python at> writes:
>>>On Aug 12, 1:37=3DA0pm, Thomas Jollans <tho... at> wrote:
>>>> On Tuesday 10 August 2010, it occurred to kj to exclaim:
>>>> > I'm looking for a module that implements "persistent lists": objects
>>>> > that behave like lists except that all their elements are stored
>>>> > on disk. =3DA0IOW, the equivalent of "shelves", but for lists rather
>>>> > than a dictionaries.
>>> . . .
>>>> You could simply use pickle to save the data every once in a while.
>>>That is a very reasonable solution.
>> Sorry I don't follow. =C2=A0Some sample code would be helpful.

>I would assume something along the lines of (untested):

>from pickle import dump


>class PersistentList(list):
>    def __init__(self, filepath):
>        self.filepath =3D filepath
>        self._mod_count =3D 0

>    def save(self):
>        with open(self.filepath, 'w') as f:
>            dump(self, f)
>        self._mod_count =3D 0

>    def append(self, *args, **kwds):
>        super(PersistentList, self).append(*args, **kwds)
>        self._mod_count +=3D 1
>        if self._mod_count >=3D MOD_THRESHOLD:
>        # obviously time-since-last-dump or other
>        #   more sophisticated metrics might be used instead

Even though it is saved periodically to disk, it looks like the
whole list remains in memory all the time?  (If so, it's not what
I'm looking for; the whole point of saving stuff to disk is to keep
the list's memory footprint low.)


More information about the Python-list mailing list