[Tutor] Shelve & immutable objects

[eryksun]

On Wed, Jan 1, 2014 at 11:43 AM, Keith Winston <keithwins at gmail.com> wrote:
> Thanks Danny, I don't understand the re-persisted part, but I'll look into
> it.

Shelf.close calls Shelf.sync to flush the cache. Here's what it does:

    >>> print(inspect.getsource(shelve.Shelf.sync))
        def sync(self):
            if self.writeback and self.cache:
                self.writeback = False
                for key, entry in self.cache.items():
                    self[key] = entry
                self.writeback = True
                self.cache = {}
            if hasattr(self.dict, 'sync'):
                self.dict.sync()

If the writeback cache is enabled, then loading a key also caches the
item in a dict. `sync` writes all of the cached items back and clears
the cache (for which it should be using `self.cache.clear()`).

It also syncs the underlying database if it has a `sync` attribute,
which the gdbm database [1] requires. Per its man page, "gdbm defaults
to no-sync mode" [2]. So actually the "f" flag for opening in fast
mode is obsolete [3].

    1. If you're using a Debian-family Linux, install the
       packages python-gdbm and python3-gdbm.
    2. http://linux.die.net/man/3/gdbm
    3. http://docs.python.org/3/library/dbm#dbm.gnu.open

Have a look at what gets stored in the database. First lets create a
database and add a list:

    >>> import shelve, dbm
    >>> sh = shelve.open('class_shelve')
    >>> sh['alist'] = [1, 2, 3]

You could leave the Shelf open and use its underlying `sh.dict`, but
I'll reopen the database instead:

    >>> sh.close()
    >>> db = dbm.open('class_shelve')
    >>> data = db['alist']
    >>> data
    b'\x80\x03]q\x00(K\x01K\x02K\x03e.'

shelve serialized the list to a byte string with the pickle protocol.
This is a program that instructs the pickle VM to build the list. You
can manually load the list with pickle.loads:

    >>> import pickle
    >>> pickle.loads(data)
    [1, 2, 3]

You can disassemble the pickle with pickletools.dis, if you're curious:

    >>> import pickletools
    >>> pickletools.dis(data)
        0: \x80 PROTO      3
        2: ]    EMPTY_LIST
        3: q    BINPUT     0
        5: (    MARK
        6: K        BININT1    1
        8: K        BININT1    2
       10: K        BININT1    3
       12: e        APPENDS    (MARK at 5)
       13: .    STOP
    highest protocol among opcodes = 2