Testing myself on a large fasta file, I find far faster performance with igzip, but also far lower compression rate (than gzip -6, the default). 

My gzipped fasta is nearly half the size of igzip at its -3 maximum. I haven't tried 'gzip -3' to compare that size.

This limitation seems to limit this greatly as a potential drop-in replacement in current version.

On Mon, Aug 17, 2020, 9:47 PM Ben Rudiak-Gould <benrudiak@gmail.com> wrote:
On Mon, Aug 17, 2020 at 8:02 AM Steven D'Aprano <steve@pearwood.info> wrote:
Perhaps I have misunderstood, but isn't this a pure implementation
change, with no user visible API changes and backward compatible output?

According to the documentation [1], it only supports compression levels 0-3. They're supposed to be comparable in ratio to zlib's levels 0-3. I found benchmarks [2] of an older version that only had compression level 1, which shows its ratio being quite a bit worse than zlib's level 1, but maybe they've improved it.

The library interface seems similar, but it isn't drop-in compatible. It doesn't appear to have equivalents of inflateCopy and deflateCopy, which are exposed by Python's standard binding. There may be other missing features that I didn't notice.

The streams it produces are of course standard-compliant, and decompression works with any standard-compliant stream, and is probably always faster than zlib.



_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-leave@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GLI3YMMXVWNADGIEVKRCM5KYJSR4INW4/
Code of Conduct: http://python.org/psf/codeofconduct/