key/value store optimized for disk storage

[Steve Howell]

On May 3, 11:59 pm, Paul Rubin <no.em... at nospam.invalid> wrote:
> Steve Howell <showel... at yahoo.com> writes:
> >     compressor = zlib.compressobj()
> >     s = compressor.compress("foobar")
> >     s += compressor.flush(zlib.Z_SYNC_FLUSH)
>
> >     s_start = s
> >     compressor2 = compressor.copy()
>
> I think you also want to make a decompressor here, and initialize it
> with s and then clone it.  Then you don't have to reinitialize every
> time you want to decompress something.

Makes sense.  I believe I got that part correct:

  https://github.com/showell/KeyValue/blob/master/salted_compressor.py

> I also seem to remember that the first few bytes of compressed output
> are always some fixed string or checksum, that you can strip out after
> compression and put back before decompression, giving further savings in
> output size when you have millions of records.

I'm pretty sure this happens for free as long as the salt is large
enough, but maybe I'm misunderstanding.