On Mon, 17 Aug 2020 08:49:23 -0000
"Ruben Vorderman"
Dear python developers,
As a bioinformatician I work a lot with gzip-compressed data. Recently I discovered Intel's Storage Acceleration Libraries at https://github.com/intel/isa-l. These people implemented the DEFLATE and INFLATE algorithms in assembly language. As a result it is much faster than zlib.
I have posted a few benchmarks in this python bug https://bugs.python.org/issue41566. (I just discovered bugs.python.org is the wrong place for feature requests. I am sorry, I am still learning about the proper way of doing this things, as this is my first feature proposal). The TLDR is that it can speed up compression by 5x while speeding up compression by 3x compared to standard gzip.
Isa-l is bsd-3-clause licensed and as such I see no licensing issues when using it in CPython.
In any case, it should be simple enough to post a package on PyPI that exposes the desired wrapper APIs. Regards Antoine.