key/value store optimized for disk storage

[Dan Stromberg]

On Thu, May 3, 2012 at 11:03 PM, Paul Rubin <no.email at nospam.invalid> wrote:

> > Sort of as you suggest, you could build a Huffman encoding for a
> > representative run of data, save that tree off somewhere, and then use
> > it for all your future encoding/decoding.
>
> Zlib is better than Huffman in my experience, and Python's zlib module
> already has the right entry points.
>
> Isn't zlib kind of dated?  Granted, it's newer than Huffman, but there's
been bzip2 and xz since then, among numerous others.

Here's something for xz:
http://stromberg.dnsalias.org/svn/xz_mod/trunk/
An xz module is in the CPython 3.3 alphas - the above module wraps it if
available, otherwise it uses ctypes or a pipe to an xz binary..

And I believe bzip2 is in the standard library for most versions of CPython.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20120503/aafc6477/attachment.html>