[Distutils] Changing the "install hooks" mechanism for PEP 426

[PJ Eby]

On Thu, Aug 15, 2013 at 7:16 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> But if we're only going to validate it via hooks, why not have the "mapping
> of names to export specifiers" just be a recommended convention for
> extensions rather than a separate exports field?

I guess I didn't explain it very well, because that's roughly what I
meant: a single namespace for all "extensions", structured as a
mapping from group names to submappings whose keys can be arbitrary
values, but whose values must then be either a string or a JSON
object, and if it's a string, then it should be an export specifier.

To put it another way, I'm saying something slightly stronger than a
recommended convention: making it a requirement that strings at that
level be import specifiers, and only allowing mappings as an
alternative.  In that way, there is a minimum level of validation
possible for the majority of extensions *by default*, without needing
an explicit validator declared.

To put it another way, it ensures that there's a kind of lingua franca
or lowest-common denominator that lets somebody understand what's
going on in most extensions, without having to understand a new
*structural* schema for every extension group.  (Just a *syntactical*
one)


> As an extension, pydist.extension_hooks would also be non-conventional,
> since it would define a new namespace, where extension names map to an
> export group of hooks. A separate export group per hook would be utterly
> unreadable.

If you already know what keys go in an entry point group, there's a
good chance you're doing it wrong.  Normally, the whole point of the
group is that the keys are defined by the publisher, not the consumer.
 The normal pattern is that the consumer names the group (representing
a hook), and the publishers name the extensions (representing
implementations for the hook).

I don't see how it makes it unreadable, but then I think in terms of
the ini syntax or setup.py syntax for defining entry points, which is
all very flat.  IMO, creating a second-level data structure for this
doesn't make a whole lot of sense, because now you're nesting
something.

I'm not even clear why you need separate registrations for the
different hooks anyway; ISTM a single hook with an event parameter is
sufficient.  Even if it weren't, I'd be inclined to just make the
information part of the key in that case, e.g.

[pydist.extension_listeners]
preinstall:foo.bar = some.module:hook

This sort of thing is very flat and easy to express in a simple
configuration syntax, which we really shouldn't lose sight of.  It's
just as easy to have write a syntax validator as a structure
validator, but if you start with structures then you have to
back-figure a syntax.  I'd very much like it to be easy to define a
simple flat syntax that's usable for 90%+ of extension use cases...
which means I'd rather not see the PEP make up its own data structures
when they're not actually needed.

Don't get me wrong, I'm okay with allowing JSON structures for
extensions in place of export strings, but I don't think there's been
a single use case proposed as yet that actually *works better* as a
data structure.

If you need to do something like have a bunch of i18n/l10n resource
definitions with locales and subpaths and stuff like that...  awesome.
 That's something that might make a lot of sense for JSON.  But when
the ultimate point of the data structure is to define an importable
entry point, and the information needed to identify it can be put into
a relatively short human readable string, ISTM that the One Obvious
Way to do it is something like a setuptools entry point -- i.e. a
basic key-value pair in a consumer-defined namespace, mapping a
semantically-valued name to an importable object.

And *most* use cases for extensions, that I'm aware of, fit that bill.
 You have to be doing something pretty complex to need anything more
complicated, *and* there has to be a possibility that you're going to
avoid importing the related code or putting in on sys.path, or else
you don't actually *save* anything by putting it in the metadata.

IOW, if you're going to have to import it anyway, there is no point to
putting it in the metadata; you might as well import it.  The only
things that make sense to put in metadata for these things are data
that tells you whether or not you need to import it.  Generally, this
means keys, not values, in other words.  (Which is why l10n and
scripts make sense to not be entry points: at the time you use them,
you're not importing 'em.)


>That's why I'm still inclined to make this one a separate top
> level field: *installers* have to know how to bootstrap the hook system, and
> I like the symmetry of separate, relatively flat, publication and
> subscription interfaces.

I don't really see the value of a separate top-level field, but then
that's because I don't see anything at all special about these hooks
that demands something more sophisticated than common entry points.
AFAICT it's a YAGNI.