Sequential Object Store

[GZ]

Hi All,

I need to store a large number of  large objects to file and then
access them sequentially. I am talking about a few thousands of
objects and each with size of a few hundred kilobytes, and total file
size a few gigabytes. I tried shelve, but it is not good at
sequentially accessing the data. In essence, shelve.keys() takes
forever.

I am wondering if there is a module that can persist a stream of
objects without having to load everything into memory. (For this
reason, I think Pickle is out, too, because it needs everything to be
in memory.)

Thanks,
GZ