Major update to the metadata 2.0 draft PEPs
I've just published the first draft of the metadata 2.0 spec that moves all of the fields that aren't part of the core metadata or potentially needed for dependency resolution out to a separate "standard metadata extensions" PEP. I think this makes PEP 426 itself substantially less overwhelming, and also provides convenient names to refer to the various other subsets of the metadata. The new PEP defining the standard extensions is PEP 459. Metadata 2.0: http://www.python.org/dev/peps/pep-0426/ Versioning: http://www.python.org/dev/peps/pep-0440/ Standard Extensions: http://www.python.org/dev/peps/pep-0459/ You can see the full deltas for PEP 426 and PEP 440 here: http://hg.python.org/peps/rev/d18629859154 Or slightly more incremental deltas in the BitBucket repo: https://bitbucket.org/pypa/pypi-metadata-formats/overview With this update, I believe PEP 440 is extremely close to acceptance, as I have tried to resolve a number of problems Donald noted while working on Warehouse, as well as the perennial problem of how to deal with versioning of packages where Linux distro vendors (or other system integrators) have applied additional patches. The PEP 440 changes: - added the "integrator suffix" (essentially an extra release segment, separated from the upstream version number by "-"). It is effectively ignored by most of the matching operators, but sorts as a higher version than the version with no suffix. - allowed date based versions for most purposes, but still reject them for the "~=" compatible version operator. The implied comparison operator for date based versions is ">=". - clarified that zeroes and leading zeroes are permitted and that they should be sorted like numbers - based on discussions with some of the Red Hat security folks, I tweaked the direct URL references to *always* require a hash in the URL, even when using HTTPS or SSH. PEP 426 and PEP 459 are further away from finalisation, but we're definitely getting closer to something I'm happy with. The PEP 426 changes: - many of the fields are gone, having moved to PEP 459 extensions - this also allowed the notion of "essential dependency metadata" (with it's own separate filename) to go away - the extension format changed to require a submapping - extensions are now versioned (with an implied 1.0 if not specified) - as with direct URLs in PEP 440, source URLs are now required to contain a hash PEP 459 overview: - standard extensions using the "python.*" namespace - the details, project, exports and commands extensions just lift several fields out of PEP 426 - the integrator extension is new, and matches the project extension structure. It allows redistributors like a Linux distro or a Python vendor to add their own info without overwriting the upstream project metadata - the new metadata_hooks extension replaces the install hooks design in earlier drafts of PEP 426. In this new scheme, Twisted, for example, would define an appropriate export group hook in order to be told about any new extensions that exported Twisted plugins and take appropriate action In this revised metadata model, distributions *will* trigger their own postinstall metadata hooks, but won't trigger their postuninstall hooks. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, Dec 21, 2013 at 9:46 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I've just published the first draft of the metadata 2.0 spec that moves all of the fields that aren't part of the core metadata or potentially needed for dependency resolution out to a separate "standard metadata extensions" PEP.
Yay!
I think this makes PEP 426 itself substantially less overwhelming, and also provides convenient names to refer to the various other subsets of the metadata.
Indeed!
Metadata 2.0: http://www.python.org/dev/peps/pep-0426/ Versioning: http://www.python.org/dev/peps/pep-0440/ Standard Extensions: http://www.python.org/dev/peps/pep-0459/
I have only been skimming these so far, will comment more later, but I just want to mention that for standard extensions, what is the rationale for not defining e.g. commands as exports? Or for that matter, defining hooks as exports? Both commands and hooks have a payload that's the same as an export payload, i.e., a reference to an importable object. I think it'd be good for them to be defined in terms of exports in order to reduce duplication of concepts and implementations, as well as providing a PEP-defined example of how to use export groups and export names effectively. (Also, noting the mapping of current script export namespaces to the new metadata would be helpful for people implementing translation tools from setuptools metadata.) Last but not least, some of the export examples should use dotted post-module names (e.g. "some.module:RandomClass.a_class_method") to highlight that qualified names can be used there. (Yes, the spec *says* qualified names, but it's the sort of thing that people overlook when the examples are all of unqualified simple names.)
On 1 Jan 2014 10:33, "PJ Eby" <pje@telecommunity.com> wrote:
On Sat, Dec 21, 2013 at 9:46 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I've just published the first draft of the metadata 2.0 spec that moves all of the fields that aren't part of the core metadata or potentially needed for dependency resolution out to a separate "standard metadata extensions" PEP.
Yay!
I think this makes PEP 426 itself substantially less overwhelming, and also provides convenient names to refer to the various other subsets of the metadata.
Indeed!
Metadata 2.0: http://www.python.org/dev/peps/pep-0426/ Versioning: http://www.python.org/dev/peps/pep-0440/ Standard Extensions: http://www.python.org/dev/peps/pep-0459/
I have only been skimming these so far, will comment more later, but I just want to mention that for standard extensions, what is the rationale for not defining e.g. commands as exports? Or for that matter, defining hooks as exports? Both commands and hooks have a payload that's the same as an export payload, i.e., a reference to an importable object. I think it'd be good for them to be defined in terms of exports in order to reduce duplication of concepts and implementations, as well as providing a PEP-defined example of how to use export groups and export names effectively.
I believe it was due to the extra layer of nesting they needed - using multiple parallel export groups with different final elements in their dotted names didn't feel right. I guess that indicates a flaw in my initial design for the export definitions though - I agree it would be better if commands and hooks could be cleanly defined within the exports mechanism, rather than needing separate custom metadata extensions. I'll take another look at using either parallel export groups, or else allowing subdivision of export groups as a general feature.
(Also, noting the mapping of current script export namespaces to the new metadata would be helpful for people implementing translation tools from setuptools metadata.)
Last but not least, some of the export examples should use dotted post-module names (e.g. "some.module:RandomClass.a_class_method") to highlight that qualified names can be used there. (Yes, the spec *says* qualified names, but it's the sort of thing that people overlook when the examples are all of unqualified simple names.)
Yeah, those are both good ideas. Cheers, Nick.
On Wed, 1/1/14, Nick Coghlan <ncoghlan@gmail.com> wrote:
I guess that indicates a flaw in my initial design for the export definitions though - I agree it would be better if commands and hooks could be cleanly defined within the exports mechanism, rather than needing separate custom metadata extensions.
+1. Regards, Vinay Sajip
On Wed, Jan 1, 2014 at 9:39 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 1 Jan 2014 10:33, "PJ Eby" <pje@telecommunity.com> wrote:
I have only been skimming these so far, will comment more later, but I just want to mention that for standard extensions, what is the rationale for not defining e.g. commands as exports? Or for that matter, defining hooks as exports? Both commands and hooks have a payload that's the same as an export payload, i.e., a reference to an importable object. I think it'd be good for them to be defined in terms of exports in order to reduce duplication of concepts and implementations, as well as providing a PEP-defined example of how to use export groups and export names effectively.
I believe it was due to the extra layer of nesting they needed - using multiple parallel export groups with different final elements in their dotted names didn't feel right.
I guess that indicates a flaw in my initial design for the export definitions though - I agree it would be better if commands and hooks could be cleanly defined within the exports mechanism, rather than needing separate custom metadata extensions.
If it's a flaw, I'd say it's in my original design of entry points. ;-) Basically, I wanted a way to do -- without XML -- what Eclipse does with its "extensions" and "extension points" machinery. I went with a "flat (with dots) is better than nested" concept. To me, though, this doesn't look terribly complicated (using entry points syntax): [python.exports.after_install] ComfyChair.plugins = ComfyChair.plugins:install_hook [python.exports.after_uninstall] ComfyChair.plugins = ComfyChair.plugins:install_hook Nor this: [python.extensions.after_install] python.exports = pip.export_group_hooks:run_install_hooks python.commands = pip.command_hook:install_wrapper_scripts [python.extensions.after_uninstall] python.exports = pip.export_group_hooks:run_uninstall_hooks (Also, adding hooks to *validate* extensions and exports at build and/or install time might be handy.) Finally, note that if the typical usecase is to define *both* an install and uninstall hook, then it might be simpler to just define the hook once, as an object with 'after_install' and 'after_uninstall' methods. This avoids the need to register a hook in two groups, and in the simplest case people can just make them both module-level functions and list the module as the export.
On 2 Jan 2014 03:59, "PJ Eby" <pje@telecommunity.com> wrote:
On Wed, Jan 1, 2014 at 9:39 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 1 Jan 2014 10:33, "PJ Eby" <pje@telecommunity.com> wrote:
I have only been skimming these so far, will comment more later, but I just want to mention that for standard extensions, what is the rationale for not defining e.g. commands as exports? Or for that matter, defining hooks as exports? Both commands and hooks have a payload that's the same as an export payload, i.e., a reference to an importable object. I think it'd be good for them to be defined in terms of exports in order to reduce duplication of concepts and implementations, as well as providing a PEP-defined example of how to use export groups and export names effectively.
I believe it was due to the extra layer of nesting they needed - using multiple parallel export groups with different final elements in their dotted names didn't feel right.
I guess that indicates a flaw in my initial design for the export definitions though - I agree it would be better if commands and hooks
could
be cleanly defined within the exports mechanism, rather than needing separate custom metadata extensions.
If it's a flaw, I'd say it's in my original design of entry points. ;-) Basically, I wanted a way to do -- without XML -- what Eclipse does with its "extensions" and "extension points" machinery. I went with a "flat (with dots) is better than nested" concept.
To me, though, this doesn't look terribly complicated (using entry points syntax):
[python.exports.after_install] ComfyChair.plugins = ComfyChair.plugins:install_hook
[python.exports.after_uninstall] ComfyChair.plugins = ComfyChair.plugins:install_hook
Nor this:
[python.extensions.after_install] python.exports = pip.export_group_hooks:run_install_hooks python.commands = pip.command_hook:install_wrapper_scripts
[python.extensions.after_uninstall] python.exports = pip.export_group_hooks:run_uninstall_hooks
(Also, adding hooks to *validate* extensions and exports at build and/or install time might be handy.)
Finally, note that if the typical usecase is to define *both* an install and uninstall hook, then it might be simpler to just define the hook once, as an object with 'after_install' and 'after_uninstall' methods. This avoids the need to register a hook in two groups, and in the simplest case people can just make them both module-level functions and list the module as the export.
Ah, true - people could provide them as either functions or class methods, and if we add a new one, only the list of permitted names (and their signatures) needs to change. And splitting commands into python.commands and python.commands.gui export groups would cover the export compatible cases. Since identifying "prebuilt commands" in the metadata is a new feature without any particularly compelling justification at this point, I'll likely just leave it out for 2.0. Cheers, Nick.
participants (3)
-
Nick Coghlan -
PJ Eby -
Vinay Sajip