Re: [Distutils] Proposal: using /etc/os-release in the "platform tag" definition for wheel files
There are still use cases for distro-specific wheels, though -- some examples include Raspbian wheels (manylinux1 is x86/x86-64 only), Alpine Linux wheels (manylinux1 is glibc only), internal deploys that want to build on Ubuntu 14.04 and deploy on 14.04 and don't need the hassle of generating manylinux-style binaries but would like a more meaningful platform tag than "linux", and for everyone who wants to extend wheel metadata to allow dependencies on external distro packages then having distro-specific wheels is probably a necessary first step.
-n On Jun 22, 2016 09:49, "Noah Kantrowitz" noah@coderanger.net wrote:
Manylinux has mostly replaced it as that covers the platforms 99% of people worry about. The tooling for manylinux is more complex than this would have been, but sunk cost etc etc and now that we have it might as well save everyone some headache.
--Noah
On Jun 22, 2016, at 8:51 AM, Nathaniel Smith njs@pobox.com wrote:
I believe the status is that there's general consensus that something
like this would be useful, but there's no one who is currently actively working in it.
On Jun 22, 2016 5:53 AM, "Vitaly Kruglikov" vkruglikov@numenta.com
wrote:
There have been no updates in over a year. Has this effort died, or
transitioned to another medium? Thx
Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On Jun 22, 2016, at 12:21, Nathaniel Smith njs@pobox.com wrote: There are still use cases for distro-specific wheels, though -- some examples include Raspbian wheels (manylinux1 is x86/x86-64 only), Alpine Linux wheels (manylinux1 is glibc only), internal deploys that want to build on Ubuntu 14.04 and deploy on 14.04 and don't need the hassle of generating manylinux-style binaries but would like a more meaningful platform tag than "linux", and for everyone who wants to extend wheel metadata to allow dependencies on external distro packages then having distro-specific wheels is probably a necessary first step.
If we want to treat distros as first-class deployment targets I think being able to use their platform features in a way that's compatible with PyPI is an important next step. However, wheel tags might be insufficient here; the main appeal of creating distro-specific wheels is being able to use distro-specific features, but those typically come along with specific package dependencies as well, and we don't have a way to express those yet.
I think this is worthwhile to do, but figuring out a way to do it is probably going to take a lot of discussion.
-glyph
On Jun 22, 2016, at 5:38 PM, Glyph glyph@twistedmatrix.com wrote:
On Jun 22, 2016, at 12:21, Nathaniel Smith <njs@pobox.com mailto:njs@pobox.com> wrote: There are still use cases for distro-specific wheels, though -- some examples include Raspbian wheels (manylinux1 is x86/x86-64 only), Alpine Linux wheels (manylinux1 is glibc only), internal deploys that want to build on Ubuntu 14.04 and deploy on 14.04 and don't need the hassle of generating manylinux-style binaries but would like a more meaningful platform tag than "linux", and for everyone who wants to extend wheel metadata to allow dependencies on external distro packages then having distro-specific wheels is probably a necessary first step.
If we want to treat distros as first-class deployment targets I think being able to use their platform features in a way that's compatible with PyPI is an important next step. However, wheel tags might be insufficient here; the main appeal of creating distro-specific wheels is being able to use distro-specific features, but those typically come along with specific package dependencies as well, and we don't have a way to express those yet.
I don’t think these two things need to be bound together. People can already today depend on platform specific things just by not publishing wheels. Adding these tags naturally follows that, where people would need to manually install items from the OS before they used them. Adding some mechanism for automating this would be a good, further addition, but I think they are separately useful (and even more useful and powerful when combined).
I think this is worthwhile to do, but figuring out a way to do it is probably going to take a lot of discussion.
-glyph
Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
— Donald Stufft
On Jun 22, 2016, at 2:42 PM, Donald Stufft donald@stufft.io wrote:
On Jun 22, 2016, at 5:38 PM, Glyph glyph@twistedmatrix.com wrote:
On Jun 22, 2016, at 12:21, Nathaniel Smith njs@pobox.com wrote: There are still use cases for distro-specific wheels, though -- some examples include Raspbian wheels (manylinux1 is x86/x86-64 only), Alpine Linux wheels (manylinux1 is glibc only), internal deploys that want to build on Ubuntu 14.04 and deploy on 14.04 and don't need the hassle of generating manylinux-style binaries but would like a more meaningful platform tag than "linux", and for everyone who wants to extend wheel metadata to allow dependencies on external distro packages then having distro-specific wheels is probably a necessary first step.
If we want to treat distros as first-class deployment targets I think being able to use their platform features in a way that's compatible with PyPI is an important next step. However, wheel tags might be insufficient here; the main appeal of creating distro-specific wheels is being able to use distro-specific features, but those typically come along with specific package dependencies as well, and we don't have a way to express those yet.
I don’t think these two things need to be bound together. People can already today depend on platform specific things just by not publishing wheels. Adding these tags naturally follows that, where people would need to manually install items from the OS before they used them. Adding some mechanism for automating this would be a good, further addition, but I think they are separately useful (and even more useful and powerful when combined).
I could see an argument for maybe building support into Pip but disallowing them on PyPI until we feel comfortable with the UX. That doesn't add much over existing private index support though.
--Noah
On Jun 22, 2016, at 2:42 PM, Donald Stufft donald@stufft.io wrote:
On Jun 22, 2016, at 5:38 PM, Glyph glyph@twistedmatrix.com wrote:
On Jun 22, 2016, at 12:21, Nathaniel Smith njs@pobox.com wrote: There are still use cases for distro-specific wheels, though -- some examples include Raspbian wheels (manylinux1 is x86/x86-64 only), Alpine Linux wheels (manylinux1 is glibc only), internal deploys that want to build on Ubuntu 14.04 and deploy on 14.04 and don't need the hassle of generating manylinux-style binaries but would like a more meaningful platform tag than "linux", and for everyone who wants to extend wheel metadata to allow dependencies on external distro packages then having distro-specific wheels is probably a necessary first step.
If we want to treat distros as first-class deployment targets I think being able to use their platform features in a way that's compatible with PyPI is an important next step. However, wheel tags might be insufficient here; the main appeal of creating distro-specific wheels is being able to use distro-specific features, but those typically come along with specific package dependencies as well, and we don't have a way to express those yet.
I don’t think these two things need to be bound together. People can already today depend on platform specific things just by not publishing wheels. Adding these tags naturally follows that, where people would need to manually install items from the OS before they used them. Adding some mechanism for automating this would be a good, further addition, but I think they are separately useful (and even more useful and powerful when combined).
I could see an argument for maybe building support into Pip but disallowing them on PyPI until we feel comfortable with the UX. That doesn't add much over existing private index support though.
--Noah
They're definitely separate; there's a draft PEP Tennessee and I started at PyCon AU 2014 which would make a good basis for such an effort.
-Rob
On 23 June 2016 at 09:42, Donald Stufft donald@stufft.io wrote:
On Jun 22, 2016, at 5:38 PM, Glyph glyph@twistedmatrix.com wrote:
On Jun 22, 2016, at 12:21, Nathaniel Smith njs@pobox.com wrote:
There are still use cases for distro-specific wheels, though -- some examples include Raspbian wheels (manylinux1 is x86/x86-64 only), Alpine Linux wheels (manylinux1 is glibc only), internal deploys that want to build on Ubuntu 14.04 and deploy on 14.04 and don't need the hassle of generating manylinux-style binaries but would like a more meaningful platform tag than "linux", and for everyone who wants to extend wheel metadata to allow dependencies on external distro packages then having distro-specific wheels is probably a necessary first step.
If we want to treat distros as first-class deployment targets I think being able to use their platform features in a way that's compatible with PyPI is an important next step. However, wheel tags might be insufficient here; the main appeal of creating distro-specific wheels is being able to use distro-specific features, but those typically come along with specific package dependencies as well, and we don't have a way to express those yet.
I don’t think these two things need to be bound together. People can already today depend on platform specific things just by not publishing wheels. Adding these tags naturally follows that, where people would need to manually install items from the OS before they used them. Adding some mechanism for automating this would be a good, further addition, but I think they are separately useful (and even more useful and powerful when combined).
I think this is worthwhile to do, but figuring out a way to do it is probably going to take a lot of discussion.
-glyph
Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
— Donald Stufft
Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On Jun 22, 2016, at 14:42, Donald Stufft donald@stufft.io wrote:
I don’t think these two things need to be bound together. People can already today depend on platform specific things just by not publishing wheels. Adding these tags naturally follows that, where people would need to manually install items from the OS before they used them. Adding some mechanism for automating this would be a good, further addition, but I think they are separately useful (and even more useful and powerful when combined).
OK, fair point. I don't disagree with any of that. As long as everyone has a clear idea that this is part of a phased approach so the wheel tags should be ultimately upwards compatible with something that does dependencies, this all sounds pretty good (and an incremental approach with benefits at each stage is more likely to succeed anyhow).
-glyph
-
Donald Stufft -
Glyph -
Nathaniel Smith -
Noah Kantrowitz -
Robert Collins