[Distutils] Changing the "install hooks" mechanism for PEP 426

[Vinay Sajip]

PJ Eby <pje <at> telecommunity.com> writes:

> I guess I didn't explain it very well, because that's roughly what I
> meant: a single namespace for all "extensions", structured as a
> mapping from group names to submappings whose keys can be arbitrary
> values, but whose values must then be either a string or a JSON
> object, and if it's a string, then it should be an export specifier.

Why should the keys be completely arbitrary? I can't see a reason for this;
the current constraint of the "prefixed name" seems sufficient. What would
relaxing this constraint make possible which otherwise isn't?

On the values: an export specifier is just a more human-friendly version of
a dict with module/content/extra keys. While of course the uses of
importables in this area is well established, what specific use cases are we
enabling by allowing arbitrary JSON? It certainly would clutter the metadata
and render it less human-readable, and the only thing it provides is a dict
which could be expressed in an importable form
"mypackage.mymodule:path.to.a.dict". Of course this incurs an import penalty
for access to the data, but if the publisher is concerned about this, they
can minimise that overhead by arranging their package hierarchy
appropriately. What kinds of use cases would require that the data be fully
expressed in the metadata to avoid import overhead?

> To put it another way, I'm saying something slightly stronger than a
> recommended convention: making it a requirement that strings at that
> level be import specifiers, and only allowing mappings as an
> alternative.

I'd like to understand the use cases which allowing mappings here would
facilitate, that need to avoid importing to access the mapping.

> If you already know what keys go in an entry point group, there's a
> good chance you're doing it wrong.  Normally, the whole point of the
> group is that the keys are defined by the publisher, not the consumer.
>  The normal pattern is that the consumer names the group (representing
> a hook), and the publishers name the extensions (representing
> implementations for the hook).

But the general form of the keys needs to be agreed to some extent between
the consumer and publisher. Otherwise, the consumer doesn't know how to
interpret the values of those keys. Of course a consumer can get all of the
key/value entries exported by a dist or all dists for a specific group, but
then what do they do with it if they don't know what the individual entries
mean?
 
> which means I'd rather not see the PEP make up its own data structures
> when they're not actually needed.

+1 - identification of the needs should come before specific proposals to
address them.

> Don't get me wrong, I'm okay with allowing JSON structures for
> extensions in place of export strings, but I don't think there's been
> a single use case proposed as yet that actually *works better* as a
> data structure.

The forms are equivalent, modulo an import penalty which only occurs for
actual use and not for just scanning to see what's available.

> Way to do it is something like a setuptools entry point -- i.e. a
> basic key-value pair in a consumer-defined namespace, mapping a
> semantically-valued name to an importable object.
> 
> And *most* use cases for extensions, that I'm aware of, fit that bill.
>  You have to be doing something pretty complex to need anything more
> complicated, *and* there has to be a possibility that you're going to
> avoid importing the related code or putting in on sys.path, or else
> you don't actually *save* anything by putting it in the metadata.
> IOW, if you're going to have to import it anyway, there is no point to
> putting it in the metadata; you might as well import it.

Agreed, so I would say that we need to identify these use cases before
saying that arbitrary mappings should be allowed as values.

Regards,

Vinay Sajip