On Wed, 23 Sep 2020 at 11:09, David Mertz email@example.com wrote:
It's hard to see where packaging would have any advantage with brotli or zstd over lzma. XZ is more widely used, and package size seems to dominate speed. There are definitely some intermediate compression levels where both brotli and zstd are significantly faster, but not at the higher levels where lzma does as well or better.
Is there a concrete need here, or just an abstract point that compression of packages shouldn't be outside the stdlib?
Honestly, if you really want compression size over everything else, PPM is going to beat the LZ based approaches. But being ungodly slow and using tons of memory.
The discussion over on the Packaging discourse channel is ongoing, and hasn't reached any conclusion yet. But yes, there have been lots of debates over what's the best compression method. That debate triggered this request, but it doesn't indicate a specific need for these compression methods. I can't speak directly for Omer, but I assume the intention was to explore what's possible/likely, to inform the discussion of wheel compression with some concrete information about what we can reasonably assume might be in the stdlib.
I don't have a personal stake here - I'd likely never use brotli or zstd unless a file format that I needed to process used them, and for any use case *other* than packaging, I don't currently care if the support is in the stdlib or not. If either format becomes commonly used, then I'd argue for them being in the stdlib, but right now they seem relatively uncommon in my personal experience.