[Tutor] using shelve

[John Fouhy]

On 22/06/07, chrispython at mac.com <chrispython at mac.com> wrote:
> I created a shelf called 'myshelf' and put two objects in it, a string and a list. When I open the shelf I get:
>
> >>> d=shelve.open('/Users/development/Desktop/myshelf')
> >>> d.keys()
> ['dir1', 'dir2']
> >>> d
> {'dir2': '/Users/development/Desktop/RSSReaderApp/RSS.db', 'dir1': ['.DS_Store', '.localized', 'access the latest version', 'Python Quick Reference.webloc', 'rssdb', 'workspace']}
>
> It seems that when you use shelve.open(), it actually brings the entire shelf dictionary into memory, accessible through d.

Well ... all that tells me is that when you ask python for a string
representation of a shelf, it reads the entire thing.

> What if you had 100 objects in myshelf, or 1000 or 100,000? Wouldn't it get bogged down?

If you try to print out the whole thing, probably.  Let's try some test:

>>> import shelve
>>> d = shelve.open('test.shelf')
>>> for i in range(1000):
...  d[str(i)] = 'x'*i
...
>>> d.close()

Morpork:~/tmp repton$ ls -l test.shelf
-rw-r--r--   1 repton  repton       1M Jun 22 14:23 test.shelf

First, we'll measure how long it takes python to open the shelf using
shelve.open:

Morpork:~/tmp repton$ python -m timeit -s 'import shelve' 'd =
shelve.open("test.shelf")'
1000 loops, best of 3: 1.95 msec per loop

Now we'll measure how long it takes python to open a shelf and turn it
into a string:

Morpork:~/tmp repton$ python -m timeit -s 'import shelve' 'd =
shelve.open("test.shelf")' 'str(d)'
10 loops, best of 3: 51.5 msec per loop

So, that's about a factor of 25 difference.

HTH!

-- 
John.