manylinux1 guidelines for zlib?
Hello, Surprisingly, the manylinux1 spec doesn't seem to include the zlib in the list of known-to-be-available libraries (are there GNU/Linux systems out there without a zlib installed?). Since I'm assuming several packages already had a need for that, is there a recommended way to link in the zlib as part of a manylinux1 wheel? Would you recommend static linking with a private version, or dynamic linking? Regards Antoine.
-----Original Message----- From: Antoine Pitrou <antoine@python.org> Sent: Monday, September 3, 2018 6:51 AM To: distutils-sig@python.org Subject: [Distutils] manylinux1 guidelines for zlib?
Hello,
Surprisingly, the manylinux1 spec doesn't seem to include the zlib in the
Since zlib is a dependency of python, the assumption has to be that it is already present. I think you can look at the python docs for which versions of zlib you should be dynamically linking against. The docs state "There are known incompatibilities between the Python module and versions of the zlib library earlier than 1.1.3; 1.1.3 has a security vulnerability, so we recommend using 1.1.4 or later." So if I were linking against zlib, I would link against 1.1.4. However, I don't know if zlib has a stable abi... so take that with a pinch of salt. list
of known-to-be-available libraries (are there GNU/Linux systems out there without a zlib installed?).
Since I'm assuming several packages already had a need for that, is there a recommended way to link in the zlib as part of a manylinux1 wheel? Would you recommend static linking with a private version, or dynamic linking?
Regards
Antoine. -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-leave@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils- sig@python.org/message/ZZG6GL3XTBLBJXSITYHEXMFKN43EREB7/
On Sep 4, 2018, at 6:06 AM, Alex Walters <tritium-list@sdamon.com> wrote:
Since zlib is a dependency of python, the assumption has to be that it is already present.
It is technically an optional dependency of Python, though I don’t think you can install wheels without zlib present since wheels are zip files and generally include compression. Although maybe it could work in niche scenarios if someone built wheels using ZIP_STORED instead of ZIP_DEFLATE?
Isn’t zlib only required for compression? It is my impression that zipfile’s decompressor is pure Python, and only depends on zlib if the archive is encrypted (but wheels are never encrypted). zlib also does not provide decompression at its core; for that you need zlib/contrib/minizip, but I don’t think CPython depends on minizip either. zlib is technically optional, and not even required to installed wheels.
On 05/9/2018, at 19:35, Donald Stufft <donald@stufft.io> wrote:
On Sep 4, 2018, at 6:06 AM, Alex Walters <tritium-list@sdamon.com <mailto:tritium-list@sdamon.com>> wrote:
Since zlib is a dependency of python, the assumption has to be that it is already present.
It is technically an optional dependency of Python, though I don’t think you can install wheels without zlib present since wheels are zip files and generally include compression. Although maybe it could work in niche scenarios if someone built wheels using ZIP_STORED instead of ZIP_DEFLATE? -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-leave@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/F...
On Sep 5, 2018, at 9:30 AM, Tzu-ping Chung <uranusjr@gmail.com> wrote:
Isn’t zlib only required for compression? It is my impression that zipfile’s decompressor is pure Python, and only depends on zlib if the archive is encrypted (but wheels are never encrypted).
zlib also does not provide decompression at its core; for that you need zlib/contrib/minizip, but I don’t think CPython depends on minizip either.
zlib is technically optional, and not even required to installed wheels.
I don’t think that’s accurate, if you look at zipfile.py, it does: https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f... <https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f...> Which attempts to import the zlib module, and if it fails sets zlib to None. Then later one this is used: https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f... <https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f...> Which doesn’t guard the expression at all, so I believe if you attempt to unzip a wheel that uses ZIP_STORED without the zlib module built, then you’ll get an AttributeError, None type does not have decompressobj exception. The Python stdlib zlib module does not appear to have any fallback at all if the zlib library is not available and a recent enough version.
I see. Thanks for the clarification!
On 05/9, 2018, at 22:54, Donald Stufft <donald@stufft.io> wrote:
On Sep 5, 2018, at 9:30 AM, Tzu-ping Chung <uranusjr@gmail.com <mailto:uranusjr@gmail.com>> wrote:
Isn’t zlib only required for compression? It is my impression that zipfile’s decompressor is pure Python, and only depends on zlib if the archive is encrypted (but wheels are never encrypted).
zlib also does not provide decompression at its core; for that you need zlib/contrib/minizip, but I don’t think CPython depends on minizip either.
zlib is technically optional, and not even required to installed wheels.
I don’t think that’s accurate, if you look at zipfile.py, it does:
https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f... <https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f...>
Which attempts to import the zlib module, and if it fails sets zlib to None. Then later one this is used:
https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f... <https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f...>
Which doesn’t guard the expression at all, so I believe if you attempt to unzip a wheel that uses ZIP_STORED without the zlib module built, then you’ll get an AttributeError, None type does not have decompressobj exception.
The Python stdlib zlib module does not appear to have any fallback at all if the zlib library is not available and a recent enough version.
Updating on this. It seems the manylinux1 tooling may automatically link and bundle an old zlib version if you link dynamically against it. That can be counter-productive... Regards Antoine.
participants (4)
-
Alex Walters -
Antoine Pitrou -
Donald Stufft -
Tzu-ping Chung