[Tutor] using shelve

[chrispython at mac.com]

Sorry it took me so long to get back - from your posts and my experimentation I can see that when you access one item in the shelve dictionary, it only gets the one item, not all of them. I am going to use shelve, and only refactor or change if performance becomes an issue - which I don't see happening for a long time.
On Thursday, June 21, 2007, at 10:27PM, "John Fouhy" <john at fouhy.net> wrote:
>On 22/06/07, chrispython at mac.com <chrispython at mac.com> wrote:
>> I created a shelf called 'myshelf' and put two objects in it, a string and a list. When I open the shelf I get:
>>
>> >>> d=shelve.open('/Users/development/Desktop/myshelf')
>> >>> d.keys()
>> ['dir1', 'dir2']
>> >>> d
>> {'dir2': '/Users/development/Desktop/RSSReaderApp/RSS.db', 'dir1': ['.DS_Store', '.localized', 'access the latest version', 'Python Quick Reference.webloc', 'rssdb', 'workspace']}
>>
>> It seems that when you use shelve.open(), it actually brings the entire shelf dictionary into memory, accessible through d.
>
>Well ... all that tells me is that when you ask python for a string
>representation of a shelf, it reads the entire thing.
>
>> What if you had 100 objects in myshelf, or 1000 or 100,000? Wouldn't it get bogged down?
>
>If you try to print out the whole thing, probably.  Let's try some test:
>
>>>> import shelve
>>>> d = shelve.open('test.shelf')
>>>> for i in range(1000):
>...  d[str(i)] = 'x'*i
>...
>>>> d.close()
>
>Morpork:~/tmp repton$ ls -l test.shelf
>-rw-r--r--   1 repton  repton       1M Jun 22 14:23 test.shelf
>
>First, we'll measure how long it takes python to open the shelf using
>shelve.open:
>
>Morpork:~/tmp repton$ python -m timeit -s 'import shelve' 'd =
>shelve.open("test.shelf")'
>1000 loops, best of 3: 1.95 msec per loop
>
>Now we'll measure how long it takes python to open a shelf and turn it
>into a string:
>
>Morpork:~/tmp repton$ python -m timeit -s 'import shelve' 'd =
>shelve.open("test.shelf")' 'str(d)'
>10 loops, best of 3: 51.5 msec per loop
>
>So, that's about a factor of 25 difference.
>
>HTH!
>
>-- 
>John.
>
>