refining the idea of entrypoints and their metadata
Hi, while toying with the entrypoint system, i repeatedly ran into the need of having additional metadata prior to importing In Plugins that only handle certain filetypes/extensions/mimetypes might profit from the additional metadata (while also defering imports) The same goes for my library anyvc that needs to know what directories to check for, however usually the directory checking is a lot faster than doing the imports (and implementing custom lazy importing is pain). So i propose supporting to store additional metadata along with the entrypoints. the current data is stored in the form of a ini file a la: [group] namea = import [extras] nameb = import [another group] ... Which basically translates to the following data entries per entrypoint: :group: the group of that entrypoint :name: the name :import: the default import :extras: the setuptools extra dependencies Now i'd like to add custom entries to that, and i see various ways to store those * list of json objects includ (strongly prefered by me) * adding [group:name] sections right after each group section that encode the additional metadata (see examples) This would allow basic uses like:: [wiki.renderer] creole = foo.bar:CreoleRenderer [creole] [wiki.renderer:creole] mimetypes: wiki/creole fnmaps: *.creole *.wiki or:: [pygments.lexer] Brainfuck = bf:BfLexer [pygments.lexer:brainfuck] mimetypes: brainfuck?! filenames: *.bf *.brainfuck aliases: bf Additionaly the load method of the Entrypoint class could grow a optional parameter for alternative imports That would allow listings like:: [anyvc.backend] subversion = anyvc.subversion [subversion] mercurial = anyvc.mercurial [mercurial] [anyvc.backend:subversion] aliases: svn workdir_adm_dir: .svn/ workdir_has_paths: .svn/entries .svn/props/ repo_has_paths: format hooks/ db/ conf/ locks/ workdir: anyvc.subversion.workdir:SubversionWorkdir repo: anyvc.subversion.repo:SubversionRepository [anyvc.backend:mercurial] aliases: hg workdir_adm_dir: .hg workdir_has_paths: .hg/requires .hg/store/ repo_has_paths: .hg/requires .hg/store/ workdir: anyvc.mercurial.workdir:HgWorkdir repo: anyvc.mercurial.repo:HgRepository Then one could just choose specific imports by passing the new name of the import to the load function like:: workdir_type = ep.load('workdir') repo_type = ep.load('repo') As you can see the above is a pretty rough idea and in need of refinements. The advantage over just having sets of Entrypoints with group, name, import is that this metadata extension provides a much more pleasant entry to medium sized feature and removes the need to import for various metadata checks (in some of my cases, importing entrypoints completely dominated the execution times) I also played with the idea of having separate entrypoints for metadata and modules, but there are many cases where that kind of use seems unpractical or really just unpleasant to me. An additional item of interest is having multiple Entrypoints which do the same thing but, are implemented using different dependencies (and i want them ordered by some kind of preference/priority). (but i suppose in case of doubt i could implement those as sort by a metadata key) Please Discuss! Regards Ronny
At 11:24 PM 1/11/2010 +0100, Ronny Pfannschmidt wrote:
Hi,
while toying with the entrypoint system, i repeatedly ran into the need of having additional metadata prior to importing
In Plugins that only handle certain filetypes/extensions/mimetypes might profit from the additional metadata (while also defering imports)
The same goes for my library anyvc that needs to know what directories to check for, however usually the directory checking is a lot faster than doing the imports (and implementing custom lazy importing is pain).
So i propose supporting to store additional metadata along with the entrypoints.
Just FYI, there is nothing stopping you from implementing something like this with an egg_info writer plugin and a custom setup argument: http://peak.telecommunity.com/DevCenter/setuptools#adding-setup-arguments http://peak.telecommunity.com/DevCenter/setuptools#adding-new-egg-info-files You could then use the pkg_resources discovery APIs to find distributions: http://peak.telecommunity.com/DevCenter/PkgResources#getting-or-creating-dis... And then pull out the metadata via the API: http://peak.telecommunity.com/DevCenter/PkgResources#imetadataprovider-metho... This is exactly how standard entry points are implemented now, so you can use this as a guide. The i18n support for Chandler uses a similar approach, bundling a resources.ini in the egg_info metadata, that lists message catalogs and other localization resources included in the relevant plugin. (Chandler plugins can provide translations for strings in other Chandler plugins, so that e.g. you can distribute a multi-plugin localization plugin.) (Btw, these comments should not be construed as saying a more sophisticated system is a bad idea or anything like that, I'm just pointing out that you need not wait for someone else to implement this for you, as the hooks to do it on your own are documented and ready to use. And perhaps you can then promote your implementation as a defacto or dejure standard. ;-) )
On Tue, 2010-01-12 at 01:28 -0500, P.J. Eby wrote:
At 11:24 PM 1/11/2010 +0100, Ronny Pfannschmidt wrote:
Hi,
while toying with the entrypoint system, i repeatedly ran into the need of having additional metadata prior to importing
In Plugins that only handle certain filetypes/extensions/mimetypes might profit from the additional metadata (while also defering imports)
The same goes for my library anyvc that needs to know what directories to check for, however usually the directory checking is a lot faster than doing the imports (and implementing custom lazy importing is pain).
So i propose supporting to store additional metadata along with the entrypoints.
Just FYI, there is nothing stopping you from implementing something like this with an egg_info writer plugin and a custom setup argument:
http://peak.telecommunity.com/DevCenter/setuptools#adding-setup-arguments http://peak.telecommunity.com/DevCenter/setuptools#adding-new-egg-info-files
You could then use the pkg_resources discovery APIs to find distributions:
http://peak.telecommunity.com/DevCenter/PkgResources#getting-or-creating-dis...
And then pull out the metadata via the API:
http://peak.telecommunity.com/DevCenter/PkgResources#imetadataprovider-metho...
This is exactly how standard entry points are implemented now, so you can use this as a guide.
This is a good inspiration on getting started.
The i18n support for Chandler uses a similar approach, bundling a resources.ini in the egg_info metadata, that lists message catalogs and other localization resources included in the relevant plugin. (Chandler plugins can provide translations for strings in other Chandler plugins, so that e.g. you can distribute a multi-plugin localization plugin.)
(Btw, these comments should not be construed as saying a more sophisticated system is a bad idea or anything like that, I'm just pointing out that you need not wait for someone else to implement this for you, as the hooks to do it on your own are documented and ready to use. And perhaps you can then promote your implementation as a defacto or dejure standard. ;-) )
I just wanted discussion first in order to get rid of initial stupid ideas - and your hint to the extension system was pretty helpfull in destroying the stupid idea of needing to hack into entrypoint parsing directly. I'll start to implement my current needs for setuptools/distribute 0.6 and distribute 0.7 and post for peer-review once i get to a acceptable point. Regards Ronny
participants (2)
-
P.J. Eby -
Ronny Pfannschmidt