Pickle diffs anyone?
Someone recently made a cool proposal for utilizing diffs to save space taken by old versions in the Zope object database: http://www.zope.org/Members/jim/ZODB/ReverseDiffVersioning To make this work, we need a good way of diffing pickles. I thought maybe someone here would have some good suggestions. I do think that the topic is sort of interesting (for some definition of "interesting" ;). The page above is a Wiki page. (Wiki is awesome. If you haven't seen it before, check out http://joyful.com/zwiki/ZWiki.) If you are a member of zope.org, you can edit the page directly, which would be fine with me. :) Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Jim Fulton writes:
To make this work, we need a good way of diffing pickles.
Jim, If the basic requirement is for a binary diff facility, perhaps you should look into XDelta; I think that's available as a C library as well as a command line tool, so you should be able to hook it in fairly easily. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> Corporation for National Research Initiatives
Jim> Someone recently made a cool proposal for utilizing diffs to save Jim> space taken by old versions in the Zope object database: Jim> http://www.zope.org/Members/jim/ZODB/ReverseDiffVersioning Jim> To make this work, we need a good way of diffing pickles. Fred already mentioned a candidate library to do diffs. If that works, the only other thing I think you'd need to do is guarantee that dicts are pickled in a consistent fashion, probably by sorting the keys before enumerating them. -- Skip Montanaro, skip@mojam.com, http://www.mojam.com/, http://www.musi-cal.com/ "We have become ... the stewards of life's continuity on earth. We did not ask for this role... We may not be suited to it, but here we are." - Stephen Jay Gould
participants (3)
-
Fred L. Drake, Jr.
-
Jim Fulton
-
Skip Montanaro