Let's put it this way. If you can only support 3 compression algorithms in
the stdlib, which there would you choose? If only 4? If only 10?
Each one is concrete maintenance work. There's nothing *wrong* with any of
them, and someone uses each of the top 10 or 50. But some kind of cut-off
of usefulness vs burden is necessary.
Paul's case of packaging needing stdlib support is plausible. I'm not quite
convinced even by that though. Just like we have ensurepip, it would be
perfectly possible to make whizbang-compression the first dependency of
every other package, but package whizbang itself using plain tar.
However, HTTP2 is absolutely something that's fine for PyPI. That really
doesn't need to be stdlib.
On Wed, Sep 23, 2020, 12:26 AM Omer Katz
I pointed out a use case for Brotli & HTTP2 as a concrete example for why it'd be more convenient to include brotli as a module. I'm sure there are other cases I haven't thought about.
I don't understand why LZMA should be included while zstd or brotli shouldn't. What's the actual policy here?
בתאריך יום ד׳, 23 בספט׳ 2020 ב-13:09 מאת David Mertz <email@example.com >:
On Tue, Sep 22, 2020 at 11:55 PM Paul Moore
The point of this request is that Python's packaging infrastructure is looking at what compression we use for wheels - the current compression is suboptimal for huge binaries like tensorflow. Packaging is in a unique situation, because it *cannot* use external libraries
It's hard to see where packaging would have any advantage with brotli or zstd over lzma. XZ is more widely used, and package size seems to dominate speed. There are definitely some intermediate compression levels where both brotli and zstd are significantly faster, but not at the higher levels where lzma does as well or better.
Is there a concrete need here, or just an abstract point that compression of packages shouldn't be outside the stdlib?
Honestly, if you really want compression size over everything else, PPM is going to beat the LZ based approaches. But being ungodly slow and using tons of memory.
-- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.