How to Buffer Serialized Objects to Disk

[Peter Otten]

Scott McCarty wrote:

> Been digging ever since I posted this. I suspected that the response might
> be use a database. I am worried I am trying to reinvent the wheel. The
> problem is I don't want any dependencies and I also don't need persistence
> program runs. 

I don't think sqlite3 counts as a dependency these days.

> I kind of wanted to keep the use of petit very similar to
> cat, head, awk, etc. But, that said, I have realized that if I provide the
> analysis features as an API, you very well, might want persistence between
> runs.
> 
> What about using an array inside a shelve?
> 
> Just got done messing with this in python shell:
> 
> import shelve
> 
> d = shelve.open(filename="/root/test.shelf", protocol=-1)
> 
> d["log"] = ()
> d["log"].append("test1")
> d["log"].append("test2")
> d["log"].append("test3")
> 
> Then, always interacting with d["log"], for example:
> 
> for i in d["log"]:
>     print i
> 
> Thoughts?

That won't save you any memory as the whole value (the complete list) is 
unpickled once you say d["log"].