Shelve problem: why so big?

[Matt Gerrans]

> I have an idea, but considering the fact that I just learned python
> yesterday, it probably won't help...

That comment made me chuckle.   (actually, it is pretty impressive that you
learned Python yesterday and are already answering questions -- you must be a
quick learner!)

As for the original post, my personal opinion is that with this much data, it
is time to consider using a database.   Or at the very least, you should come
up with a scheme to manage and partition in to multiple smaller chunks.   Even
if it worked with shelve module, wouldn't the performance be pretty awful?

The PyWin database modules are pretty handy and the dbm module can be used on
Unix platforms.   I don't know about other platforms, though.   There might be
even better database access available...

Even if you don't use a database, if all you are storing is a bunch of floats,
you don't even need shelve.   It is easy enough to write them to (and read
from) a file on your own with a loop, isn't it?

- Matt