Memory usage steadily going up while pickling objects

[Giorgos Tzampanakis]

I have a program that saves lots (about 800k) objects into a shelve
database (I'm using sqlite3dbm for this since all the default python dbm
packages seem to be unreliable and effectively unusable, but this is
another discussion).

The process takes about 10-15 minutes. During that time I see memory usage
steadily rising, sometimes resulting in a MemoryError. Now, there is a
chance that my code is keeping unneeded references to the stored objects,
but I have debugged it thoroughly and haven't found any.

So I'm beginning to suspect that the pickle module might be keeping an
internal cache of objects being pickled. Is this true?

-- 
Real (i.e. statistical) tennis and snooker player rankings and ratings:
http://www.statsfair.com/