save tuple of simple data types to disk (low memory foot print)

Gelonida N gelonida at gmail.com
Fri Oct 28 16:47:42 EDT 2011


Hi,

I would like to save many dicts with a fixed amount of keys
tuples to a file  in a memory efficient manner (no random, but only
sequential access is required)

As the keys are the same for each entry  I considered converting them to
tuples.

The tuples contain only strings, ints (long ints) and floats (double)
and the data types for each position within the tuple are fixed.

The fastest and simplest way is to pickle the data or to use json.
Both formats however are not that optimal.


I could store ints and floats with pack. As strings have variable length
I'm not sure how to save them efficiently
(except adding a length first and then the string.

Is there already some 'standard' way or standard library to store
such data efficiently?

Thanks in advance for any suggestion.




More information about the Python-list mailing list