[Distutils] Q about best practices now (or near future)

[Nick Coghlan]

On 22 Jul 2013 01:46, "PJ Eby" <pje at telecommunity.com> wrote:
>
> On Sat, Jul 20, 2013 at 10:54 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > Extras should just be a way to ask "are these optional dependencies
present
> > on this system?", without needing to worry about how they got there.
>
> Technically, they are a way to ask "can you get this for me?", since
> pkg_resources' API allows you to specify an installer callback when
> you ask to load an entry point.  This means that an installer tool can
> dynamically obtain any extras it needs, not just check for their
> installation.
>
> To put it another way, it's not "exported only if extra is available",
> it's "exported, but make sure you have this first."  A subtle
> difference, but important to the original use cases (see below).

Ah, yes, I see the distinction (and it does make this notion conceptually
simpler).

>
>
> > For now, I'll switch export specifiers back to the concise
> > "modulename:qualname" entry point format and add "Do we need to
> > support the exported-only-if-extra-is-available feature?" as an open
> > question. My current thinking is that the point you made about script
> > wrappers (putting the wrapper in separate distribution and depending
> > on that from an extra) applies to other plugins as well.
>
> Now that I'm thinking about it some more, one of the motivating use
> cases for extras in entry points was startup performance in
> plugin-heavy GUI applications like Chandler.  The use of extras allows
> for late-loading of additions to sys.path.  IOW, it's intended more
> for a situation where not only are the entry points imported late, but
> you also want as few plugins as possible on sys.path to start with, in
> order to have fast startup.

I'm working with Eric Snow on a scheme that I hope will allow
module-specific path entries that aren't processed at interpreter startup
and never get added to sys.path at all (even if you import the module).
Assuming we can get it to work the way I hope (which is still a "maybe" at
this point in time), it should then be possible to backport it to earlier
versions as a metaimporter.

> The other use case is similar, in that a plugin-heavy environment with
> self-upgrading abilities can defer *installation* of parts of a
> plug-in until it is actually used.  (Which is why EntryPoint objects
> have a .require() method separate from .load() - you can loop over a
> relevant set of entry points to pre-test or pre-ensure that they're
> all available and dependencies are installed before importing any of
> them, even though .load() will also do that for a single entry point.)

OK, so as Daniel suggested, it's more like an export/entry-point specific
"requires" field, but limited to the extras of the current distribution.

> For the specific case of the meta build system itself, these use cases
> may be moot.  For the overall use of exports, however, the use cases
> are still valuable for plugin-heavy apps.  (Specifically, applications
> that use lots of plugins written by different people, and don't want
> to have to import everything at startup.)
>
> Indeed, this is the original use case for exports in the first place:
> it's a plugin system that doesn't require importing any plugins until
> you actually need a particular plugin's functionality.  Extras just
> expand that slightly to "don't require installing things or putting
> them on sys.path until you need their functionality".

OK, I understand the use case now. If I can come up with a relatively
simple way to explain it, I'll keep it in the proposed metadata, otherwise
I'll leave it to metadata extensions to handle the more sophisticated
version where an export depends on an extra.

Cheers,
Nick.

>
> Heck, if pip itself were split into two distributions, one of which
> were a command line script declared with an extra, pointing into the
> second distribution, it'd have dynamic bootstrapping.   (Were it not
> for the part where it would need pip available to *do* the
> bootstrapping, of course.  ;-) )
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130722/8d818e39/attachment.html>