migration path for entry_points?
ok, so PEP459 has exports: http://www.python.org/dev/peps/pep-0459/#the-exports-extension and distlib seems to be implementing them. and "exports" seem to be entry points 2.0. so theoretically, when PEP426/PEP459 becomes live.... what does that mean for setuptools-based projects? (I understand that we might not have a clear idea now, but just trying to narrow down the possibilities) 1) the setuptools "entry_points" keyword (and pkg_resources api) gets re-implemented to understand PEP459 exports? or 2) "entry_points" keeps the old implementation, and possibly tools are written to handle the old and new metadata? or 3) "entry_points" are stranded, and consumers have to rewrite setup.py files to use some new keyword that understands PEP459 or 4) something altogether different... as for why I'm asking, pip itself is considering command extensions, so it's a direct practical matter for us. Marcus
It's nice to have that split up into lots of more focused documents. The comment I have is that I think entry points sometimes have multiple values with the same name (a list or multidict instead of a simple dict) although I'm not sure how common that is... Tools should just do automatic conversion of entry_points.txt to the new metadata format when .dist-info is in use. On Tue, Dec 31, 2013 at 3:05 PM, Marcus Smith <qwcode@gmail.com> wrote:
ok, so PEP459 has exports: http://www.python.org/dev/peps/pep-0459/#the-exports-extension and distlib seems to be implementing them.
and "exports" seem to be entry points 2.0.
so theoretically, when PEP426/PEP459 becomes live....
what does that mean for setuptools-based projects? (I understand that we might not have a clear idea now, but just trying to narrow down the possibilities)
1) the setuptools "entry_points" keyword (and pkg_resources api) gets re-implemented to understand PEP459 exports? or 2) "entry_points" keeps the old implementation, and possibly tools are written to handle the old and new metadata? or 3) "entry_points" are stranded, and consumers have to rewrite setup.py files to use some new keyword that understands PEP459 or 4) something altogether different...
as for why I'm asking, pip itself is considering command extensions, so it's a direct practical matter for us.
Marcus
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On 1 Jan 2014 06:58, "Daniel Holth" <dholth@gmail.com> wrote:
It's nice to have that split up into lots of more focused documents.
Yeah, I was really happy with how breaking things up into multiple extensions worked out. I'm still pondering the question of whether or not to aim for the updated installation database format (with a check for an sqlite DB in each path entry before looking directly at the metadata files) in the metadata 2.0 time frame. It's a potentially preferable approach to speeding up runtime lookups over extracting additional per-distribution data files. Maybe it's worth at least defining that as a *goal* and then deciding the time frame for when we want to get there?
The comment I have is that I think entry points sometimes have multiple values with the same name (a list or multidict instead of a simple dict) although I'm not sure how common that is...
If the current format allows repeated keys with different values, we may need to allow lists as values in addition to strings in the PEP 459 approach.
Tools should just do automatic conversion of entry_points.txt to the new metadata format when .dist-info is in use.
Yep, as with other fields, my aim is to create a format that can be derived from the existing setuptools metadata. That way, PyPI can publish it even for projects that only upload metadata using the current setuptools formats. Cheers, Nick.
On Tue, Dec 31, 2013 at 3:05 PM, Marcus Smith <qwcode@gmail.com> wrote:
ok, so PEP459 has exports: http://www.python.org/dev/peps/pep-0459/#the-exports-extension and distlib seems to be implementing them.
and "exports" seem to be entry points 2.0.
so theoretically, when PEP426/PEP459 becomes live....
what does that mean for setuptools-based projects? (I understand that we might not have a clear idea now, but just trying to narrow down the possibilities)
1) the setuptools "entry_points" keyword (and pkg_resources api) gets re-implemented to understand PEP459 exports? or 2) "entry_points" keeps the old implementation, and possibly tools are written to handle the old and new metadata? or 3) "entry_points" are stranded, and consumers have to rewrite setup.py
files
to use some new keyword that understands PEP459 or 4) something altogether different...
as for why I'm asking, pip itself is considering command extensions, so it's a direct practical matter for us.
Marcus
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Tools should just do automatic conversion of entry_points.txt to the new metadata format when .dist-info is in use.
Yep, as with other fields, my aim is to create a format that can be derived from the existing setuptools metadata. That way, PyPI can publish it even for projects that only upload metadata using the current setuptools formats.
ok, but to be clear about the *usage* of the "entry_points" setup keyword... that's the user question. Should someone feel ok with adopting this now? It sounds like you're saying yes. setuptools users should assume either: a) new-style tool consumers of metadata will auto-convert old-style entry points b) setuptools will update the implementation of "entry_points" to produce be PEP459 compliant
let's consider 2 cases 1) someone publishes an sdist extension, SomeExtension-1.0.tar.gz up on pypi, that uses the setuptools "entry_points" keyword 2) someone publishes a wheel extension, SomeExtension-1.0.whl that has a traditional "entry_points.txt" file baked into it. and then PEP426 goes live, and pip/setuptools refactor themselves to be compliant, i.e. setuptools starts generating 2.0 metadata, and pip (via a new pkg_resources, distlib, or some other alternative) is able to process 2.0 metadata. for #1, during installation, I imagine the new setuptools would build the sdist (and any entry_point declarations) using 2.0 metadata. for #2, the pip installer would be responsible for converting "entry_points.txt" during the install (or maybe just forcing a "bdist_wheel" rebuild when it detects metadata < 2.0) On Tue, Dec 31, 2013 at 1:35 PM, Marcus Smith <qwcode@gmail.com> wrote:
Tools should just do automatic conversion of entry_points.txt to the new metadata format when .dist-info is in use.
Yep, as with other fields, my aim is to create a format that can be derived from the existing setuptools metadata. That way, PyPI can publish it even for projects that only upload metadata using the current setuptools formats.
ok, but to be clear about the *usage* of the "entry_points" setup keyword... that's the user question. Should someone feel ok with adopting this now? It sounds like you're saying yes. setuptools users should assume either: a) new-style tool consumers of metadata will auto-convert old-style entry points b) setuptools will update the implementation of "entry_points" to produce be PEP459 compliant
On 31 December 2013 22:53, Marcus Smith <qwcode@gmail.com> wrote:
for #1, during installation, I imagine the new setuptools would build the sdist (and any entry_point declarations) using 2.0 metadata.
So assuming the package consuming the entry point converts to using metadata 2.0 APIs (and does *not* try to have a compatibility mode that falls back to the old setuptools format) users would be required to use an up to date setuptools (installation with older setuptools would be unsupported). Also they would need to reinstall SomeExtension for it to be recognised by the new consumer.
for #2, the pip installer would be responsible for converting "entry_points.txt" during the install (or maybe just forcing a "bdist_wheel" rebuild when it detects metadata < 2.0)
Again requiring users to be running a suitably recent pip and/or setuptools and reinstall SomeExtension. Presumably the new setuptools and pip would have to maintain both the Metadata 2.0 format data and the old-style entry_points.txt format for a period of compatibility. We should probably define the length of that transition period as part of the plan for Metadata 2.0 going live. So there is some user impact, but it sounds like it's manageable. Cool. Paul
On 1 Jan 2014 20:58, "Paul Moore" <p.f.moore@gmail.com> wrote:
On 31 December 2013 22:53, Marcus Smith <qwcode@gmail.com> wrote:
for #1, during installation, I imagine the new setuptools would build
the
sdist (and any entry_point declarations) using 2.0 metadata.
So assuming the package consuming the entry point converts to using metadata 2.0 APIs (and does *not* try to have a compatibility mode that falls back to the old setuptools format) users would be required to use an up to date setuptools (installation with older setuptools would be unsupported). Also they would need to reinstall SomeExtension for it to be recognised by the new consumer.
for #2, the pip installer would be responsible for converting "entry_points.txt" during the install (or maybe just forcing a "bdist_wheel" rebuild when it detects metadata < 2.0)
Again requiring users to be running a suitably recent pip and/or setuptools and reinstall SomeExtension.
Presumably the new setuptools and pip would have to maintain both the Metadata 2.0 format data and the old-style entry_points.txt format for a period of compatibility. We should probably define the length of that transition period as part of the plan for Metadata 2.0 going live.
So there is some user impact, but it sounds like it's manageable. Cool.
Yeah, one of the big benefits of switching to the JSON format with a different filename was letting us distribute both the old and new metadata formats in parallel. The aim is to allow newer versions of pip and setuptools to be faster and more reliable, without breaking older versions. Cheers, Nick.
Paul
participants (4)
-
Daniel Holth -
Marcus Smith -
Nick Coghlan -
Paul Moore