On 22 Jul 2013 01:46, "PJ Eby" <pje@telecommunity.com> wrote:
On Sat, Jul 20, 2013 at 10:54 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Extras should just be a way to ask "are these optional dependencies
present
on this system?", without needing to worry about how they got there.
Technically, they are a way to ask "can you get this for me?", since pkg_resources' API allows you to specify an installer callback when you ask to load an entry point. This means that an installer tool can dynamically obtain any extras it needs, not just check for their installation.
To put it another way, it's not "exported only if extra is available", it's "exported, but make sure you have this first." A subtle difference, but important to the original use cases (see below).
Ah, yes, I see the distinction (and it does make this notion conceptually simpler).
For now, I'll switch export specifiers back to the concise "modulename:qualname" entry point format and add "Do we need to support the exported-only-if-extra-is-available feature?" as an open question. My current thinking is that the point you made about script wrappers (putting the wrapper in separate distribution and depending on that from an extra) applies to other plugins as well.
Now that I'm thinking about it some more, one of the motivating use cases for extras in entry points was startup performance in plugin-heavy GUI applications like Chandler. The use of extras allows for late-loading of additions to sys.path. IOW, it's intended more for a situation where not only are the entry points imported late, but you also want as few plugins as possible on sys.path to start with, in order to have fast startup.
I'm working with Eric Snow on a scheme that I hope will allow module-specific path entries that aren't processed at interpreter startup and never get added to sys.path at all (even if you import the module). Assuming we can get it to work the way I hope (which is still a "maybe" at this point in time), it should then be possible to backport it to earlier versions as a metaimporter.
The other use case is similar, in that a plugin-heavy environment with self-upgrading abilities can defer *installation* of parts of a plug-in until it is actually used. (Which is why EntryPoint objects have a .require() method separate from .load() - you can loop over a relevant set of entry points to pre-test or pre-ensure that they're all available and dependencies are installed before importing any of them, even though .load() will also do that for a single entry point.)
OK, so as Daniel suggested, it's more like an export/entry-point specific "requires" field, but limited to the extras of the current distribution.
For the specific case of the meta build system itself, these use cases may be moot. For the overall use of exports, however, the use cases are still valuable for plugin-heavy apps. (Specifically, applications that use lots of plugins written by different people, and don't want to have to import everything at startup.)
Indeed, this is the original use case for exports in the first place: it's a plugin system that doesn't require importing any plugins until you actually need a particular plugin's functionality. Extras just expand that slightly to "don't require installing things or putting them on sys.path until you need their functionality".
OK, I understand the use case now. If I can come up with a relatively simple way to explain it, I'll keep it in the proposed metadata, otherwise I'll leave it to metadata extensions to handle the more sophisticated version where an export depends on an extra. Cheers, Nick.
Heck, if pip itself were split into two distributions, one of which were a command line script declared with an extra, pointing into the second distribution, it'd have dynamic bootstrapping. (Were it not for the part where it would need pip available to *do* the bootstrapping, of course. ;-) )