PEP 376 proposed changes for basic plugins support
Hello, Here's a proposal to extend PEP 376 to support a basic plugins feature -- you should read PEP 376 before reading this mail It's basically Phillip's entry points, but with an activation flag, and a per-user config file. = adding a PLUGINS file = A new file called 'PLUGINS' is added to the dist-info directory at install time, and contains a list of plugins for the installed distribution. The file is a CSV file like the RECORD file, containing one plugin per line. (see the RECORD file for details on the syntax) A plugin is defined by : - an application name : a string containing a name of a group of plugins. It can be the name of the application that supports plugins, or any string. The recommended value is the distutils name of the project, as it appears at PyPI. max length: 64, chars: [a-z0-9] - a plugin type name: a string containing a name of a type of plugins. It can be combined with the application name to create one class of plugins for a given application. 64 chars [a-z0-9] -- default value 'plugin' - a name: a string containing a name for the plugin. Combined with the application name and plugin type, it defines this plugin. 64 chars [a-z0-9] - a description: a string describing the plugin. 256 chars. - a state: 1 or 0. 1 means the plugin is activated, 0 means it's not activated - a code link: a link to the plugin object (whether its a module, class, object, etc). This link is the complete import path. For example, a plugin class called "Foo" located in the "bar" module, located in the "baz" package, is noted: "baz.bar.Foo" (the project that uses plugins is responsible for iteratting, picking and loading plugins) = PLUGINS example = Here's an example of a PLUGINS file for a 'UnittestPlugins' project that implements 2 unitest2 plugins: unittest2;plugin;pep8;pep8 checker;1;unittestplugins.pep8 unittest2;plugin;debugger,Cool debugger;1;unittestplugins.debug = per-user plugins = A plugin can be activated or disable globally, but a user should be able to configure them differently. A ini-like plugins.cfg file is added per-user (its location to be defined -- its discussed in another thread) and overrides the state of the installed plugin. It provides a value for each app.type.name plugin. [plugins] unittest2.plugin.pep8 = 0 distutils2.commands.funkycommand = 0 Notice that the user can have plugins provided by distributions installed in his own per-user site packages. = Implementation = I don't want to go into great details here yet, until we get a consensus on the PLUGINS file. But basically we will add these features in pkgutil: - browse plugins and get their info. - activate/disable a plugin, by writing its state - load a plugin and return it by importing the 'code link' and in distutils2: - let the user configure if plugins are automatically activated when the project is installed - provide a end-user script to browse plugins - provide a way to define plugins statically in setup.cfg Regards Tarek -- Tarek Ziadé | http://ziade.org
On Sun, Aug 1, 2010 at 11:55 PM, Nick Coghlan <ncoghlan@gmail.com> wrote: ...
Is dealing with name conflicts left up to the application?
When an application iterates on the plugins that are supposely built for it, it will probably expect a single type of object. For instance unitest2 will want classes that are overriding its Plugin base class. So it should be able to drop objects that are not compliant with its system. Now, since the distutils id at PyPI is unique, using it for the application name in the plugin like I proposed, should also help preventing this issue.
On Sun, Aug 01, 2010 at 10:37:47PM +0200, Tarek Ziad? wrote:
The file is a CSV file
In what encoding (charset)? I quickly skimmed over PEPs 262, 241, 314 and 376, but didn't encountered any mention of the words "encoding" or "charset". Documentation for the "csv" module also doesn't provide any clear indication of encoding/charset. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
On Sun, Aug 1, 2010 at 11:58 PM, Oleg Broytman <phd@phd.pp.ru> wrote:
utf-8. I'll add this info in PEP 376 -- Thanks
-- Tarek Ziadé | http://ziade.org
On Mon, Aug 02, 2010 at 12:11:06AM +0200, Tarek Ziad? wrote:
Thank you for the clarification! Oleg. -- Oleg Broytman http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
On 01/08/2010 21:37, Tarek Ziadé wrote:
This seems fine; I mean it isn't written directly by humans or intended to be read directly by humans I guess. :-) (Users will specify plugins in the setup metadata and this will be written on install by distutils2 - right?.)
I don't think that unittest would use a distutils2 (or pkgutil) supplied API for activation. unittest needs *separate* per user and per project activation (a plugin active for a project may not be needed in other projects and so won't be enabled at the user level for example). unittest also needs plugin *configuration* and it makes sense to keep configuration and activation in the same place. The interesting part of this proposal for unittest is the plugin discovery. unittest will use the two config file solution (location of user config still to be determined) and probably a "plugins" subcommand (eventually) to manage the config file from the command line. On the other hand the activation is likely to be wanted by other projects that use plugins and it certainly doesn't hurt unittest.
Yes, great.
- activate/disable a plugin, by writing its state - load a plugin and return it by importing the 'code link'
Also great.
All sounds good to me. Michael
Regards Tarek
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
On Mon, Aug 2, 2010 at 1:11 AM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Yes, exactly
But the discovery API you will use might just simply filter out disabled plugins. In any case, if unittest2 tries to bypass this activation flag I don't see the point of having it there since its purpose is to let the end-user deactivate a plugin that might be used by several applications.
And sharing plugins across apps is a use case too: Nose could use unittest2 plugins and vice-versa. Tarek
On 02/08/2010 00:46, Tarek Ziadé wrote:
I did consider asking this but thought it was a silly question - there *must* be an API to get all plugins otherwise applications couldn't activate or deactivate them (or allow their users to do this) - and then how could users activate a non-active plugin? (I guess there could be private APIs that only distutils2 is supposed to use, or the script that you mentioned.) On the other hand if the user has deactivated a plugin through distutils2 I have no problem with it not being available to unittest.
Right. As I explained, I don't think unittest *can* use this mechanism since it can have plugins active for one project but not for another. I would really have no problem with this machinery existing, but it wouldn't be useful/usable by unittest for plugins. It sounds like it can fairly easily be bolted on as a new feature set later *anyway*, so dropping it for now simplifies the initial implementation. Wouldn't that mean that distutils2 would still need its *own* system for telling whether or not installed plugins are active? So maybe you have to build it anyway.
Hehe, well - that's a different story... Michael
Tarek
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
On Mon, Aug 2, 2010 at 1:56 AM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
yes, there will be a way to retrieve them all ...
It sounds like it can fairly easily be bolted on as a new feature set later *anyway*, so dropping it for now simplifies the initial implementation.
but then we would be back to the problem mentioned about entry points: installing projects can implicitly add a plugin and activate it, and break existing applications that iterate over entry points without further configuration. So being able to disable plugins from the beginning seems important to me
On 02/08/2010 01:03, Tarek Ziadé wrote:
I agree it sounds like an important feature - a point of control for the user or the system as you said on irc. I still don't think unittest *can* use it, but I'm quite happy with the fact that if a user deactivates a plugin through distutils2 then it is no longer *available* to unittest. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
On 2 Aug, 2010, at 2:03, Tarek Ziadé wrote:
It should be the role of the application to manage which plugins are enabled and not the library. The library can provide an API that applications can use to manage which plugins are activated, but the library should IMHO not force a specific scheme onto the application. Keeping management outside of the libraries allows applications to provide their own mechanisms for it, such as the unittest2 configuration or even a configuration screen in a GUI application. The latter is what's the most interesting to me, have a GUI application where users can optionally add functionality by adding a egg-file[*] to a plugin directory. It would be nice if I could use the stdlib plugin API for that instead having to use my own inventions. BTW. The spec seems to assume that the PLUGINS file is writeable, that needn't be true for example when packages are installed by a system tool or on multi-user systems where only sysadmins can install packages in a central site-packages location. Ronald [*] that is, a zipfile containing an installed distutils distribution
On Sun, Aug 1, 2010 at 10:37 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
After some more thoughts, ISTM that it would be better to have the plugin definitions in a new metadata field in PEP 345, rather than in the PLUGINS file. The reason is that it will allow tools and users to browse PyPI to look for plugins. The PLUGINS file can be kept only for the state value, which is not read-only. Regards, Tarek -- Tarek Ziadé | http://ziade.org
Just to add a general opinion in here: Having worked with Setuptools' entry points, and a little with some Zope pluginish systems (Products.*, which I don't think anyone liked much, and some ways ZCML is used is pluginish), I'm not very excited about these. The plugin system that causes the least confusion and yet seems to accomplish everything it needs is just listing objects in configuration -- nothing gets activated implicitly with installation, and names are Python package/object names without indirection. The only thing I'd want to add is the ability to also point to files, as a common use for plugins is adding ad hoc functionality to an application, and the overhead of package creation isn't always called for. hg for example seems both simple and general enough, and it doesn't use anything fancy. Purely for the purpose of discovery and documentation it might be helpful to have APIs, then some tool could show available plugins (especially if PyPI had a query interface for this), or at least installed plugins, with the necessary code to invoke them. *Maybe* it would make sense to generalize the discovery of plugin types, so that you can simply refer to an object and the application can determine what kind of plugin it is. But having described this, it actually doesn't seem like a useful thing to generalize. -- Ian Bicking | http://blog.ianbicking.org
On Mon, Aug 2, 2010 at 6:57 PM, Ian Bicking <ianb@colorstudy.com> wrote:
This makes it a two-step process to use a plugin: install it and then configure it correctly to have it used. I'd much prefer a one-step process and rather provide a way to not-use a plugin even if installed. The difference is e.g. with py.test that i can point users to e.g. pip install pytest-figleaf py.test --figleaf instead of also having to explain a configuration file, its location and exact content or some magic attribute variables on some classes. TBH, i'd like to have pip handle plugins, based on metadata (especially some data signaling something is a plugin of otherthing). This way i only once have to teach about "pip" and people leverage their knowledge for installing/managing plugins. best, holger
On 02/08/2010 19:05, Holger Krekel wrote:
How do you achieve this currently? Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
On Mon, Aug 2, 2010 at 8:12 PM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
it uses setuptools entrypoints, so with a setup.py param like entry_points = {'pytest11': ['pytest_figleaf = pytest_figleaf'],}, py.test's pluginmanager.py does: for ep in pkg_resources.iter_entry_points('pytest11'): # ... ep.name == "pytest_figleaf" <- left side of above "*=*" plugin = ep.load() # <- right side of above "*=*" # ... best, holger
On 02/08/2010 19:45, Holger Krekel wrote:
Right. I can't use that for unittest. With the new proposal we *could* automatically use all available plugins, but I think I prefer to only have plugins active that the user has chosen. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
On Mon, Aug 2, 2010 at 8:48 PM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
This sounds like a situation where a user has more installed than he actually asked for. I guess i am a big fan of "use virtualenv, avoid global installations" and thus don't see the point to have more installed than needed. best, holger
participants (8)
-
Holger Krekel
-
Ian Bicking
-
Michael Foord
-
Nick Coghlan
-
Oleg Broytman
-
Ronald Oussoren
-
Tarek Ziadé
-
Éric Araujo