[Python-Dev] PEP 376 proposed changes for basic plugins support

[David Cournapeau]

On Tue, Aug 3, 2010 at 11:35 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> On 03/08/2010 15:19, David Cournapeau wrote:
>>
>> On Tue, Aug 3, 2010 at 8:48 PM, Antoine Pitrou<solipsis at pitrou.net>
>>  wrote:
>>
>>>
>>> On Tue, 03 Aug 2010 10:28:07 +0200
>>> "M.-A. Lemburg"<mal at egenix.com>  wrote:
>>>
>>>>>
>>>>> Don't forget system packaging tools like .deb, .rpm, etc., which do not
>>>>> generally take kindly to updating such things.  For better or worse,
>>>>> the
>>>>> filesystem *is* our "central database" these days.
>>>>>
>>>>
>>>> I don't think that's a problem: the SQLite database would be a cache
>>>> like e.g. a font cache or TCSH command cache, not a replacement of
>>>> the meta files stored in directories.
>>>>
>>>> Such a database would solve many things at once: faster access to
>>>> the meta-data of installed packages, fewer I/O calls during startup,
>>>> more flexible ways of doing queries on the meta-data, needed for
>>>> introspection and discovery, etc.
>>>>
>>>
>>> If the cache can become stale because of system package management
>>> tools, how do you avoid I/O calls while checking that the database is
>>> fresh enough at startup?
>>>
>>
>> There is a tension between the two approaches: either you want
>> "auto-discovery", or you want a system with explicit registration and
>> only the registered plugins would be visible to the system.
>>
>>
>
> Not true. Auto-discovery provides an API for applications to tell users
> which plugins are *available* whilst still allowing the app to decide which
> are active / enabled. It still leaves full control in the hands of the
> application.

Maybe  I was not clear, but I don't understand how your statement
contradict mine. The issue is how to determine which plugins are
available: if you don't have an explicit registration, you need to
constantly restat every potential location (short of using OS specific
systems to to get notification from fs changes). The current python
solutions that I am familiar with are prohibitively computing
intensive for this reason (think about what happens when you stat
locations on NFS shares).

David