[Python-Dev] Inclusion of lz4 bindings in stdlib?

[Gregory P. Smith]

On Wed, Nov 28, 2018 at 9:52 AM Brett Cannon <brett at python.org> wrote:

> Are we getting to the point that we want a compresslib like hashlib if we
> are going to be adding more compression algorithms?
>

Lets avoid the lib suffix when unnecessary.  I used the name hashlib
because the name hash was already taken by a builtin that people normally
shouldn't be using.  zlib gets a lib suffix because a one letter name is
evil and it matches the project name. ;)  "compress" sounds nicer.

... looking on PyPI to see if that name is taken:
https://pypi.org/project/compress/ exists and is already effectively what
you are describing.  (never used it or seen it used, no idea about quality)

I don't think adding lz4 to the stdlib is worthwhile.  It isn't required
for core functionality as zlib is (lowest common denominator zip support).
I'd argue that bz2 doesn't even belong in the stdlib, but we shouldn't go
removing things.  PyPI makes getting more algorithms easy.

If anything, it'd be nice to standardize on some stdlib namespaces that
others could plug their modules into.  Create a compress in the stdlib with
zlib and bz2 in it, and a way for extension modules to add themselves in a
managed manner instead of requiring a top level name?  Opening up a
designated namespace to third party modules is not something we've done as
a project in the past though.  It requires care.  I haven't thought that
through.

-gps


>
> On Wed, 28 Nov 2018 at 08:44, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>> On Wed, 28 Nov 2018 10:28:19 +0000
>> Jonathan Underwood <jonathan.underwood at gmail.com> wrote:
>> > Hi,
>> >
>> > I have for sometime maintained the Python bindings to the LZ4
>> > compression library[0, 1]:
>> >
>> > I am wondering if there is interest in having these bindings move to
>> > the standard library to sit alongside the gzip, lzma etc bindings?
>> > Obviously the code would need to be modified to fit the coding
>> > guidelines etc.
>>
>> Personally I would find it useful indeed.  LZ4 is very attractive
>> when (de)compression speed is a primary factor, for example when
>> sending data over a fast network link or a fast local SSD.
>>
>> Another compressor worth including is Zstandard (by the same author as
>> LZ4). Actually, Zstandard and LZ4 cover most of the (speed /
>> compression ratio) range quite well. Informative graphs below:
>>
>> https://gregoryszorc.com/blog/2017/03/07/better-compression-with-zstandard/
>>
>> Regards
>>
>> Antoine.
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20181128/61c0121d/attachment.html>