Distro packagers: PEP 615 and the tzdata dependency
Hi all, One part of PEP 615 (IANA time zones) that I expected to be easily overlooked (a point about which I seem to have been right) is the recommendation that starting with Python 3.9, distros should add a dependency on the system time zone data to the Python package. Quoting the PEP:
Python distributors are encouraged to ensure that time zone data is installed alongside Python whenever possible (e.g. by declaring tzdata as a dependency for the python package).
https://www.python.org/dev/peps/pep-0615/#system-time-zone-information I am not sure what is the best way to reach the largest number of distro packagers to suggest that they add a dependency on tzdata for Python 3.9+, so I figured I'd send a message here. If you know a distro packager who does not follow this list, or you use a distro that doesn't have a dependency on tzdata for Python 3.9, please forward this e-mail to them and/or open a bug on their issue tracker. So far I know conda-forge got the dependency right from the first Python 3.9 release, Fedora has plans to fix this <https://src.fedoraproject.org/rpms/python3.10/pull-request/13>, and Christian Heimes has opened a bug on the Ubuntu launchpad <https://bugs.launchpad.net/ubuntu/+source/python3.9/+bug/1904271> for this. I will figure out how best to notify Arch Linux and do that (since that's the distro I use). I suspect this will be most important for distros that are heavily used in containers, since tzdata is a common enough dependency that it's likely to be satisfied accidentally in most full-featured user environments anyway. Thanks! Paul P.S. If there's a mailing list or other community for Python distro packagers, please let me know, because I often find that we're making decisions that affect them that they don't hear about until way down the line. I'd love to have an easy way to keep them in the loop.
Python distributors are encouraged to ensure that time zone data is installed alongside Python whenever possible (e.g. by declaring tzdata as a dependency for the python package). https://www.python.org/dev/peps/pep-0615/#system-time-zone-information I am not sure what is the best way to reach the largest number of distro packagers to suggest that they add a dependency on tzdata for Python 3.9+, so I figured I'd send a message here. If you know a distro
On Mon, 2020-11-16 at 09:32 -0500, Paul Ganssle wrote: Hi all, One part of PEP 615 (IANA time zones) that I expected to be easily overlooked (a point about which I seem to have been right) is the recommendation that starting with Python 3.9, distros should add a dependency on the system time zone data to the Python package. Quoting the PEP: packager who does not follow this list, or you use a distro that doesn't have a dependency on tzdata for Python 3.9, please forward this e-mail to them and/or open a bug on their issue tracker. So far I know conda-forge got the dependency right from the first Python 3.9 release, Fedora has plans to fix this, and Christian Heimes has opened a bug on the Ubuntu launchpad for this. I will figure out how best to notify Arch Linux and do that (since that's the distro I use). I suspect this will be most important for distros that are heavily used in containers, since tzdata is a common enough dependency that it's likely to be satisfied accidentally in most full-featured user environments anyway. Thanks! Paul P.S. If there's a mailing list or other community for Python distro packagers, please let me know, because I often find that we're making decisions that affect them that they don't hear about until way down the line. I'd love to have an easy way to keep them in the loop. Hi, In Arch Linux, tzdata is a dependency of glibc, which is part of the base[1] package that should be present on every installation. So, there is no action necessary :) We could make it an explicit dependency, but that it not necessarily required, it is up to the maintainer, whom I have notified. I would say the best approach to reach distro packages is to open a bug in their issue tracker, or to reach them via mailing list. Some of them have specific mailing lists for Python, Fedora has python-devel[2]. Maybe it would make sense to create a community mailing list for packagers? I would also suggest to in the future maybe have a "For packagers" section, or similar, in the release notes. I don't think right now there is any reasonable way packagers could have known about this new dependency unless they are involved with the Python upstream. All the current changelog does is link to PEP 615, where that information is present, but it has no mention of new dependencies. Please let me know if I've overlooked something :P [1] https://www.archlinux.org/packages/core/any/base/ [2] https://lists.fedoraproject.org/archives/list/python-devel%40lists.fedorapro... Cheers, Filipe Laíns
On 11/16/20 9:57 AM, Filipe Laíns wrote:
In Arch Linux, tzdata is a dependency of glibc, which is part of the base[1] package that should be present on every installation. So, there is no action necessary :) We could make it an explicit dependency, but that it not necessarily required, it is up to the maintainer, whom I have notified.
I opened a bug on the tracker including a patch: https://bugs.archlinux.org/task/68642?project=1&string=python I do think it makes sense to make it a direct dependency, since I personally worry that transitive dependencies may be fragile (and even if the transitive dependency is rock-solid, it doesn't hurt anything for it to exist — plus it gives you metadata about what projects are using what packages, for purposes of notifying or looking for issues). That said, it's obviously up to y'all. There's actually another thing that is probably of interest to distro packagers, which is that zoneinfo includes a compile-time configuration option: https://docs.python.org/3/library/zoneinfo.html#zoneinfo-data-compile-time-c... By default we don't know where the zoneinfo will be deployed, so we use a set of locations where it's commonly deployed. Since distro packagers know exactly where it is deployed, they can use the `--with-tzpath` configuration option to specify it at build time, to make the lookup that much faster and more accurate.
I would say the best approach to reach distro packages is to open a bug in their issue tracker, or to reach them via mailing list. Some of them have specific mailing lists for Python, Fedora has python-devel[2].
Yeah, I was mainly looking for a way to contact all of them at once, since it doesn't scale very well for me to open bugs on the trackers of every distributor. I can open bugs for my distro and distros that I care about, but even that can be pretty duplicative.
Maybe it would make sense to create a community mailing list for packagers? That sounds like a good idea to me. I would also suggest to in the future maybe have a "For packagers" section, or similar, in the release notes. I don't think right now there is any reasonable way packagers could have known about this new dependency unless they are involved with the Python upstream. All the current changelog does is link to PEP 615, where that information is present, but it has no mention of new dependencies. This also seems like a good idea to me (though I'm not entirely sure how often packagers are impacted by changes to CPython — zoneinfo may have been an unusual case).
Best, Paul
On 11/16/20 4:10 PM, Paul Ganssle wrote:
Maybe it would make sense to create a community mailing list for packagers? That sounds like a good idea to me.
I am following the Linux SIG mailing list. But it's mostly dead. https://mail.python.org/archives/list/linux-sig@python.org/ -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
That’s what I was going to suggest. I’m not doing any Debian or Ubuntu work these days, but Matthias Klose should be watching both lists, and should be able to handle the Debuntu stack. -Barry
On Nov 16, 2020, at 07:45, Miro Hrončok <mhroncok@redhat.com> wrote:
On 11/16/20 4:10 PM, Paul Ganssle wrote:
Maybe it would make sense to create a community mailing list for packagers? That sounds like a good idea to me.
I am following the Linux SIG mailing list. But it's mostly dead.
https://mail.python.org/archives/list/linux-sig@python.org/
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DF3UXOGY... Code of Conduct: http://python.org/psf/codeofconduct/
On 11/16/20 6:46 PM, Barry Warsaw wrote:
That’s what I was going to suggest. I’m not doing any Debian or Ubuntu work these days, but Matthias Klose should be watching both lists, and should be able to handle the Debuntu stack.
why is this dependency needed? The tzdata package is marked with Priority required. See Debian policy 2.5. Matthias
-Barry
On Nov 16, 2020, at 07:45, Miro Hrončok <mhroncok@redhat.com> wrote:
On 11/16/20 4:10 PM, Paul Ganssle wrote:
Maybe it would make sense to create a community mailing list for packagers? That sounds like a good idea to me.
I am following the Linux SIG mailing list. But it's mostly dead.
https://mail.python.org/archives/list/linux-sig@python.org/
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DF3UXOGY... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/23R5ZD5Z... Code of Conduct: http://python.org/psf/codeofconduct/
why is this dependency needed? The tzdata package is marked with Priority required. See Debian policy 2.5.
Matthias
Not to put words in Barry's mouth, but I took Barry's comment to be more an answer to the question of how to contact "distro packagers" as a group, more than he was taking a position about this particular issue. That said, as I mentioned on the launchpad <https://bugs.launchpad.net/ubuntu/+source/python3.9/+bug/1904271/comments/4> and elsewhere in this thread, I think that, in general, it makes sense to have an explicit dependency on tzdata as a clear indicator that we actually have a direct dependency on tzdata, even if it's not technically required because it's satisfied automatically for some reason. It would make it easier for people to find packages that depend on tzdata to notify us of issues (I've done something like this before — `tzlocal` is planning to make a backwards-incompatible change to using `zoneinfo`, and when I found out about this I dug up all the projects I could find that explicitly depend on `tzlocal` to notify them of the upcoming change). It also is less fragile. As Christian mentioned in the launchpad issue, zoneinfo is actually broken without a direct dependency on tzdata in the ubuntu docker container — possibly that's a bug in the way the ubuntu docker contianer is packaged, but it wouldn't be a problem for /us/ if python had a dependency on tzdata. As far as I can tell it also wouldn't hurt anything for this dependency to be made explicit rather than implicit. That said — it's entirely up to individual package maintainers how they want to handle their dependency metadata. The important thing I wanted to convey to distro maintainers is that PEP 615 effectively is saying that Python now depends on the system metadata package being present. If there's no need to do anything because tzdata is an implicit dependency for the entire Debian ecosystem then that's fine. Best, Paul On 11/16/20 2:15 PM, Matthias Klose wrote:
On 11/16/20 6:46 PM, Barry Warsaw wrote:
That’s what I was going to suggest. I’m not doing any Debian or Ubuntu work these days, but Matthias Klose should be watching both lists, and should be able to handle the Debuntu stack.
-Barry
On Nov 16, 2020, at 07:45, Miro Hrončok <mhroncok@redhat.com> wrote:
On 11/16/20 4:10 PM, Paul Ganssle wrote:
Maybe it would make sense to create a community mailing list for packagers? That sounds like a good idea to me. I am following the Linux SIG mailing list. But it's mostly dead.
https://mail.python.org/archives/list/linux-sig@python.org/
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DF3UXOGY... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/23R5ZD5Z... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MF4J5HIB... Code of Conduct: http://python.org/psf/codeofconduct/
On Nov 16, 2020, at 11:51, Paul Ganssle <paul@ganssle.io> wrote:
Not to put words in Barry's mouth, but I took Barry's comment to be more an answer to the question of how to contact "distro packagers" as a group, more than he was taking a position about this particular issue.
Correct! Cheers, -Barry
Good to know. I think one reason I wouldn't immediately think to send things to Linux-SIG is that for me "distro packagers" also includes people maintaining packages in the BSD, Solaris, conda-forge, homebrew, etc ecosystems. It might make sense to have a dedicated list or discourse for distro packagers, but as I don't meet the description I don't know how useful it would be for me to opine on the form it should take. Best, Paul On 11/16/20 10:45 AM, Miro Hrončok wrote:
On 11/16/20 4:10 PM, Paul Ganssle wrote:
Maybe it would make sense to create a community mailing list for packagers? That sounds like a good idea to me.
I am following the Linux SIG mailing list. But it's mostly dead.
On 16/11/2020 15.32, Paul Ganssle wrote:
Hi all,
One part of PEP 615 (IANA time zones) that I expected to be easily overlooked (a point about which I seem to have been right) is the recommendation that starting with Python 3.9, distros should add a dependency on the system time zone data to the Python package. Quoting the PEP:
Python distributors are encouraged to ensure that time zone data is installed alongside Python whenever possible (e.g. by declaring tzdata as a dependency for the python package).
https://www.python.org/dev/peps/pep-0615/#system-time-zone-information
I am not sure what is the best way to reach the largest number of distro packagers to suggest that they add a dependency on tzdata for Python 3.9+, so I figured I'd send a message here. If you know a distro packager who does not follow this list, or you use a distro that doesn't have a dependency on tzdata for Python 3.9, please forward this e-mail to them and/or open a bug on their issue tracker.
So far I know conda-forge got the dependency right from the first Python 3.9 release, Fedora has plans to fix this <https://src.fedoraproject.org/rpms/python3.10/pull-request/13>, and Christian Heimes has opened a bug on the Ubuntu launchpad <https://bugs.launchpad.net/ubuntu/+source/python3.9/+bug/1904271> for this. I will figure out how best to notify Arch Linux and do that (since that's the distro I use).
I suspect this will be most important for distros that are heavily used in containers, since tzdata is a common enough dependency that it's likely to be satisfied accidentally in most full-featured user environments anyway.
Thanks!
Paul
P.S. If there's a mailing list or other community for Python distro packagers, please let me know, because I often find that we're making decisions that affect them that they don't hear about until way down the line. I'd love to have an easy way to keep them in the loop.
Thanks Paul, would it make sense to add a packaging section to our documentation or to write an informational PEP? There is already PEP 394, which defines the "python" command on Unix-like systems. A new informational PEP could 1) assist packagers with best practices, 2) state how the CPython core team envisions packaging of Python on Unix distributions. After all there are more dependencies than "tzdata". Python has dependencies on /etc/mime.types and system trust store for the ssl module. Users also have been complaining about usability issues and bugs caused by downstream patches and packaging decisions (like venv, distutils, tkinter split into optional packages). By the way Fedora isn't affected by lack of tzdata database. Fedora's Python package has a transient dependency on tzdata. The package is always shipped in Fedora's minimal container images, too. Miro's PR makes the dependency explicit. Christian
participants (6)
-
Barry Warsaw -
Christian Heimes -
Filipe Laíns -
Matthias Klose -
Miro Hrončok -
Paul Ganssle