Shelve operations are very slow and create huge files

[Eric Wichterich]

Hello Pythonistas,

I use Python shelves to store results from MySQL-Queries (using Python 
for web scripting).
One script searches the MySQL-database and stores the result, the next 
script reads the shelve again and processes the result. But there is a 
problem: if the second script is called too early, the error "(11, 
'Resource temporarily unavailable') " occurs.
So I took a closer look at the file that is generated by the shelf: The 
result-list from MySQL-Query contains 14.600 rows with 7 columns. But, 
the saved file is over 3 MB large and contains over 230.000 lines (!), 
which seems way too much!

Following statements are used:
dbase = shelve.open(filename)
if dbase.has_key(key): #overwrite objects stored with same key
	del dbase[key]
dbase[key] = object
dbase.close()

Any ideas?

Thanks,
Eric