Environment markers: ready for prime time?
Hi! pip 7 enables wheel caching by default, which is good. Wheel caching means you can't compute dynamic dependencies any more -- i.e. things like setup( ... install_requires=[...] + (['enum34'] if sys.version_info[0] < 3 else []), ...) will cause problems. As far as I understand, you're supposed to use environment markers instead. Problem 1: where's the documentation? E.g. https://python-packaging-user-guide.readthedocs.org/en/latest/distributing/ has no mention of the word "marker". Try to google "setuptools environment marker" (and how is a user going to discover the magical keyword they need to google is "environment marker")? Try to find anything resembling documentation on the 1st page. The best I could find was http://docs.openstack.org/developer/pbr/#environment-markers which only works if you use pbr. Even the spec (https://www.python.org/dev/peps/pep-0426/#environment-markers) only shows how the markers are supposed to appear in the JSON metadata. No clue is provided for poor writers of setup.py files. I somehow discovered the syntax once (I don't remember how -- most likely kind people in #pypa spoon-fed me), but now I'm cargo-culting my existing setup.py files that already use environment markers. Problem 2: what are the minimum versions of the tools that your users must have before you can rely on environment markers? - setuptools >= 0.7 ("Added experimental environment marker support") - wheel >= 0.24 (if you have wheel 0.23 or older, environment markers are silently broken and have fun figuring out why: https://github.com/pypa/pip/issues/2870). - does the pip version matter at all? I think not; please correct me if I'm wrong. Some official answers from the hard-working PyPA visionaries would be welcome. Marius Gedminas -- I once held a little hand That made my sad heart sing. Twas the loveliest hand I'd ever held, Four Aces and a King
bdist_wheel predates marker support in setuptools and didn't support pulling markers from setup.py until 0.24.0, as noted in the wheel release notes. Older versions of bdist_wheel could read conditional requirements from a section in setup.cfg but could not translate markers from setuptools' requires.txt. Wheel's own source code demonstrates the syntax. pip uses pkg_resources to parse dependencies for wheels and an old enough pip doesn't use wheels at all. On Wed, Oct 28, 2015 at 1:15 PM Marius Gedminas <marius@gedmin.as> wrote:
Hi!
pip 7 enables wheel caching by default, which is good.
Wheel caching means you can't compute dynamic dependencies any more -- i.e. things like
setup( ... install_requires=[...] + (['enum34'] if sys.version_info[0] < 3 else []), ...)
will cause problems.
As far as I understand, you're supposed to use environment markers instead.
Problem 1: where's the documentation? E.g. https://python-packaging-user-guide.readthedocs.org/en/latest/distributing/ has no mention of the word "marker".
Try to google "setuptools environment marker" (and how is a user going to discover the magical keyword they need to google is "environment marker")? Try to find anything resembling documentation on the 1st page. The best I could find was http://docs.openstack.org/developer/pbr/#environment-markers which only works if you use pbr.
Even the spec ( https://www.python.org/dev/peps/pep-0426/#environment-markers) only shows how the markers are supposed to appear in the JSON metadata. No clue is provided for poor writers of setup.py files.
I somehow discovered the syntax once (I don't remember how -- most likely kind people in #pypa spoon-fed me), but now I'm cargo-culting my existing setup.py files that already use environment markers.
Problem 2: what are the minimum versions of the tools that your users must have before you can rely on environment markers?
- setuptools >= 0.7 ("Added experimental environment marker support")
- wheel >= 0.24 (if you have wheel 0.23 or older, environment markers are silently broken and have fun figuring out why: https://github.com/pypa/pip/issues/2870).
- does the pip version matter at all? I think not; please correct me if I'm wrong.
Some official answers from the hard-working PyPA visionaries would be welcome.
Marius Gedminas -- I once held a little hand That made my sad heart sing. Twas the loveliest hand I'd ever held, Four Aces and a King _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On Wed, Oct 28, 2015 at 7:08 PM, Marius Gedminas <marius@gedmin.as> wrote:
- does the pip version matter at all? I think not; please correct me if I'm wrong.
From what I see, pip only supports env markers in sdists since 6.0 ( https://github.com/pypa/pip/pull/2153) so I guess one can only count on env marker suport since 6.0, right? How much usage do the pre-6.0 pip versions get? Thanks, -- Ionel Cristian Mărieș, http://blog.ionelmc.ro
On 29 October 2015 at 06:08, Marius Gedminas <marius@gedmin.as> wrote:
Hi!
pip 7 enables wheel caching by default, which is good.
Wheel caching means you can't compute dynamic dependencies any more -- i.e. things like
setup( ... install_requires=[...] + (['enum34'] if sys.version_info[0] < 3 else []), ...)
will cause problems.
As far as I understand, you're supposed to use environment markers instead.
Problem 1: where's the documentation? E.g. https://python-packaging-user-guide.readthedocs.org/en/latest/distributing/ has no mention of the word "marker".
Try to google "setuptools environment marker" (and how is a user going to discover the magical keyword they need to google is "environment marker")? Try to find anything resembling documentation on the 1st page. The best I could find was http://docs.openstack.org/developer/pbr/#environment-markers which only works if you use pbr.
Even the spec (https://www.python.org/dev/peps/pep-0426/#environment-markers) only shows how the markers are supposed to appear in the JSON metadata. No clue is provided for poor writers of setup.py files.
I somehow discovered the syntax once (I don't remember how -- most likely kind people in #pypa spoon-fed me), but now I'm cargo-culting my existing setup.py files that already use environment markers.
Problem 2: what are the minimum versions of the tools that your users must have before you can rely on environment markers?
- setuptools >= 0.7 ("Added experimental environment marker support")
- wheel >= 0.24 (if you have wheel 0.23 or older, environment markers are silently broken and have fun figuring out why: https://github.com/pypa/pip/issues/2870).
- does the pip version matter at all? I think not; please correct me if I'm wrong.
Some official answers from the hard-working PyPA visionaries would be welcome.
Not an official, but ... https://github.com/pypa/pip/issues/3111 https://bugs.launchpad.net/pbr/+bug/1502692 may give some useful info for you. Don't forget that markers are broken vis-a-vis Python 2.7.10, which we need to fix asap. -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
participants (4)
-
Daniel Holth -
Ionel Cristian Mărieș -
Marius Gedminas -
Robert Collins