On Wed, Nov 28, 2018 at 9:52 AM Brett Cannon <brett@python.org> wrote:
Are we getting to the point that we want a compresslib like hashlib if we are going to be adding more compression algorithms?

Lets avoid the lib suffix when unnecessary.  I used the name hashlib because the name hash was already taken by a builtin that people normally shouldn't be using.  zlib gets a lib suffix because a one letter name is evil and it matches the project name. ;)  "compress" sounds nicer.

... looking on PyPI to see if that name is taken: https://pypi.org/project/compress/ exists and is already effectively what you are describing.  (never used it or seen it used, no idea about quality)

I don't think adding lz4 to the stdlib is worthwhile.  It isn't required for core functionality as zlib is (lowest common denominator zip support).  I'd argue that bz2 doesn't even belong in the stdlib, but we shouldn't go removing things.  PyPI makes getting more algorithms easy.

If anything, it'd be nice to standardize on some stdlib namespaces that others could plug their modules into.  Create a compress in the stdlib with zlib and bz2 in it, and a way for extension modules to add themselves in a managed manner instead of requiring a top level name?  Opening up a designated namespace to third party modules is not something we've done as a project in the past though.  It requires care.  I haven't thought that through.


On Wed, 28 Nov 2018 at 08:44, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Wed, 28 Nov 2018 10:28:19 +0000
Jonathan Underwood <jonathan.underwood@gmail.com> wrote:
> Hi,
> I have for sometime maintained the Python bindings to the LZ4
> compression library[0, 1]:
> I am wondering if there is interest in having these bindings move to
> the standard library to sit alongside the gzip, lzma etc bindings?
> Obviously the code would need to be modified to fit the coding
> guidelines etc.

Personally I would find it useful indeed.  LZ4 is very attractive
when (de)compression speed is a primary factor, for example when
sending data over a fast network link or a fast local SSD.

Another compressor worth including is Zstandard (by the same author as
LZ4). Actually, Zstandard and LZ4 cover most of the (speed /
compression ratio) range quite well. Informative graphs below:



Python-Dev mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org
Python-Dev mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-dev/greg%40krypto.org