On 20 October 2017 at 06:34, Donald Stufft <donald@stufft.io> wrote:

On Oct 19, 2017, at 4:04 PM, Donald Stufft <donald@stufft.io> wrote:

Like I said, I’m perfectly fine documenting that if you add an entry_points.txt to the .dist-info directory, that is an INI file that contains a section named “console_scripts” and define what is valid inside of the console_scripts section that it will generate script wrappers, then fine. But we should leave any other section in this entry_points.txt file as undefined in packaging terms, and point people towards setuptools for more information about it if they want to know anything more than what we need for packaging.

To be more specific here, the hypothetical thing we would be documenting/standardizing here is console entry points and script wrappers, not a generic plugin system. So console scripts would be the focus of the documentation.

We've already effectively blessed console_scripts as a standard approach: https://packaging.python.org/tutorials/distributing-packages/#entry-points

The specific problem that blessing creates is that we currently only define:

- a way for publishers to specify console_scripts via setuptools

- a way for installers to find console_scripts using pkg_resources

That's *very* similar to the problem we had with dependency declarations: only setuptools knew how to write them, and only easy_install knew how to read them.

Beyond the specific example of console_scripts, there are also multiple subecosystems where both publishers and subscribers remain locked into the setuptools/pkg_resources combination because they use entry points for their plugin management. This means that if you want to write a pytest plugin, for example, the only officially supported way to do so is to use setuptools in order to publish the relevant entry point definitions: https://docs.pytest.org/en/latest/writing_plugins.html#setuptools-entry-points

If we want to enable pytest plugin authors to use other build systems like flit, then those build systems need a defined interoperability format that's compatible with what pytest is expecting to see (i.e. entry point definitions that pkg_resources knows how to read).

We ended up solving the previous tight publisher/installer coupling problem for dependency management *not* by coming up with completely new metadata formats, but rather by better specifying the ones that setuptools already knew how to emit, such that most publishers didn't need to change anything, and even when there were slight differences between the way setuptools worked and the agreed interoperability standards, other tools could readily translate setuptools output into the standardised form (e.g. egg_info -> PEP 376 dist-info directories and wheel metadata).

The difference in this case is that:

1. entry_points.txt is already transported reliably through the whole packaging toolchain

2. It is the existing interoperability format for `console_scripts` definitions

3. Unlike setup.cfg & pyproject.toml, actual humans never touch it - it's written and read solely by software

This means that the interoperability problems we actually care about solving (allowing non-setuptools based publishing tools to specify console_scripts and other pkg_resources entry points, and allowing non-pkg_resources based consumers to read pkg_resources entry point metadata, including console_scripts) can both be solved *just* by properly specifying the existing de facto format.

So standardising on entry_points.txt isn't a matter of "because setuptools does it", it's because formalising it is the least-effort solution to what we actually want to enable: making setuptools optional on the publisher side (even if you need to publish entry point metadata), and making pkg_resources optional on the consumer side (even if you need to read entry point metadata).

I do agree that the metadata caching problem is best tackled as a specific motivating example for supporting packaging installation and uninstallation hooks, but standardising the entry points format still helps us with that: it means we can just define "python.install_hooks" as a new entry point category, and spend our energy on defining the semantics and APIs of the hooks themselves, rather than having to worry about defining a new format for how publishers will declare how to run the hooks, or how installers will find out which hooks have been installed locally.

Cheers,

Nick.

Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia