[Python-Dev] Planning on removing cache invalidation for file finders

[Nick Coghlan]

On Sun, Mar 3, 2013 at 2:16 AM, Brett Cannon <brett at python.org> wrote:
> On Sat, Mar 2, 2013 at 10:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I think you should keep it. A long running service that periodically
>> scans the importers for plugins doesn't care if modules take a few
>> extra seconds to show up, it just wants to see them eventually.
>> Installers (or filesystem copy or move operations!) have no way to
>> inform arbitrary processes that new files have been added.
>
>
> But if they are doing the scan they can also easily invalidate the caches
> before performing the scan.

"I just upgraded to Python 3.4, and now my server process isn't see new plugins"

That's a major backwards compatibility breach, and hence clearly
unacceptable in my view. Even the relatively *minor* compatibility
breach of becoming dependent on the filesystem timestamp resolution
for picking up added modules, creating a race condition between
writing the file and reading it back through the import system, has
caused people grief. When you're in a hole, the first thing to do is
to *stop digging*.

You can deprecate the heuristic if you want (and can figure out how),
but a definite -1 on removing it without at least the usual
deprecation period for backwards incompatible changes.

It may also be worth tweaking the wording of the upgrade note in the
What's New to mention the need to always invalidate the cache before
scanning for new modules if you want to reliably pick up new modules
created since the application started (at the moment the note really
only mentions it as something to do after *creating* a new module).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia