Best method for updating pickled data objects?

Rupert Scammell rupe at metro.yak.net
Thu Jul 5 16:14:26 EDT 2001


I was recently working on a project that required that a single
dictionary object be periodically written to a file via the pickle
module.  I quickly realized that the pickle.Pickler(file).dump(object)
function would write updated copies of the same data object
sequentially to the data file, rather than over-writing the older copy
of the object with new data.

To work around the problem, I wrote the following function, which
takes two arguments; dictionary_file, a string containing the name of
the file to pickle the data object in, and object, the name of the
data object to pickle.

def hash_update(dictionary_file, object):
        os.remove(dictionary_file)
        dfile = open(dictionary_file,"w")
        pickle_file = pickle.Pickler(dfile)
        pickle_file.dump(object)
        dfile.close()

Deleting and recreating the pickle file, then writing out the data
object from memory each time seems horribly inefficient to me
(especially for large data objects). There's got to be a better way to
do this.  Any suggestions would be appreciated.



More information about the Python-list mailing list