[Python-Dev] LZMA compression support in 3.3

[Nadeem Vawda]

On Sat, Aug 27, 2011 at 4:50 PM, "Martin v. Löwis" <martin at v.loewis.de> wrote:
>> The implementation will also be similar to bz2 - basic compressor and
>> decompressor classes written in C, with convenience functions and a file
>> interface implemented on top of those in Python.
>
> When I reviewed lzma, I found that this approach might not be
> appropriate. lzma has many more options and aspects that allow tuning
> and selection, and a Python LZMA library should provide the same feature
> set as the underlying C library.
>
> So I would propose that a very thin C layer is created around the C
> library that focuses on the actual algorithms, and that any higher
> layers (in particular file formats) are done in Python.

I probably shouldn't have used the word "basic" here - these classes expose all
the features of the underlying library. I was rather trying to underscore that
the rest of the module is implemented in terms of these two classes.

As for file formats, these are handled by liblzma itself; the extension module
just selects which compressor/decompressor initializer function to use depending
on the value of the "format" argument. Our code won't contain anything along the
lines of GzipFile; all of that work is done by the underlying C library. Rather,
the LZMAFile class will be like BZ2File - just a simple filter that passes the
read/written data through a LZMACompressor or LZMADecompressor as appropriate.

Cheers,
Nadeem