[Python-Dev] PEP 376 proposed changes for basic plugins support

Michael Foord fuzzyman at voidspace.org.uk
Tue Aug 3 13:09:00 CEST 2010


On 03/08/2010 09:28, M.-A. Lemburg wrote:
> P.J. Eby wrote:
>    
>> At 10:37 PM 8/2/2010 +0200, M.-A. Lemburg wrote:
>>      
>>> If that's the case, then it would be better to come up with an
>>> idea of how to make access to that meta-data available in a less
>>> I/O intense way, e.g. by having pip or other package managers update
>>> a central SQLite database cache of the data found on disk.
>>>        
>> Don't forget system packaging tools like .deb, .rpm, etc., which do not
>> generally take kindly to updating such things.  For better or worse, the
>> filesystem *is* our "central database" these days.
>>      
> I don't think that's a problem: the SQLite database would be a cache
> like e.g. a font cache or TCSH command cache, not a replacement of
> the meta files stored in directories.
>
> Such a database would solve many things at once: faster access to
> the meta-data of installed packages, fewer I/O calls during startup,
> more flexible ways of doing queries on the meta-data, needed for
> introspection and discovery, etc.
>    

Sounds good as an "optional extra" (i.e. it should be safe to completely 
delete the cache file and still have everything work) to me. If the API 
for using the feature is worked out well first this could be done as a 
behind the scenes optimisation...

>    
>> Btw, while adding PLUGINS to PEP 376 is a new proposal, it's essentially
>> another spelling of the existing entry_points.txt used by eggs; it
>> changes the format to csv instead of .ini, and adds "description" and
>> "type" fields, but drops requirements information and I'm not sure if it
>> can point to arbitrary objects the way entry_points.txt can.
>>
>> Anyway, entry_points.txt has been around enough years in the field that
>> the concept itself can't really be called "new" - it's actually quite
>> proven.  Checking
>> http://nullege.com/codes/search/pkg_resources.iter_entry_points/call , I
>> find 187 modules using just that one entry points API.
>>
>> Some projects do have more than one module loading plugins, but the
>> majority of those 187 appear to be different projects.
>>
>> Note that that's modules *loading plugins*, not plugins being
>> provided...  so the total number of PyPI projects using entry points in
>> some way is likely much higher, once you add in the plugins that these
>> 187 lookups are, well, looking up.
>>      
> setuptools entry points are just one way of doing plugins. There
> are other such systems that work well and which do not require
> any special administration or setup, simply because the application
> using the plugins defines the plugin protocol.
>    
Right, and those won't magically stop working if this proposal is 
implemented.

> Since you are into comparing numbers, you might want to count
> the number of Zope plugins that are available on PyPI and its plugin
> system has been around much longer than setuptools has been.
> I don't think that proves anything, though.
>
> I simply don't see a good reason to complicate the Python
> packaging system by trying to add a particular plugin support
> to it.
>
> Plugins are application scope features and should be treated
> as such.
>
>    
The fact is that entry points *are* widely used and solve a problem that 
you *can't* solve without some feature like this. The success of entry 
points demonstrates their utility (and you talk vaguely about 'problems' 
setuptools caused without any concrete examples - do you know of any 
*specific* difficulties with entry points?).

I doubt I will change your mind, but the bottom line is that if you 
don't like this feature you don't have to use it. For applications that 
want it (like the unittest plugin system) it will be *enormously* useful 
and *reduce* complexity for the user (by allowing simpler plugin 
management tools).

All the best,

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




More information about the Python-Dev mailing list