[Python-Dev] Planning on removing cache invalidation for file finders

[Brett Cannon]

On Sat, Mar 2, 2013 at 12:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Sun, Mar 3, 2013 at 2:16 AM, Brett Cannon <brett at python.org> wrote:
> > On Sat, Mar 2, 2013 at 10:36 AM, Nick Coghlan <ncoghlan at gmail.com>
> wrote:
> >> I think you should keep it. A long running service that periodically
> >> scans the importers for plugins doesn't care if modules take a few
> >> extra seconds to show up, it just wants to see them eventually.
> >> Installers (or filesystem copy or move operations!) have no way to
> >> inform arbitrary processes that new files have been added.
> >
> >
> > But if they are doing the scan they can also easily invalidate the caches
> > before performing the scan.
>
> "I just upgraded to Python 3.4, and now my server process isn't see new
> plugins"
>
> That's a major backwards compatibility breach, and hence clearly
> unacceptable in my view. Even the relatively *minor* compatibility
> breach of becoming dependent on the filesystem timestamp resolution
> for picking up added modules, creating a race condition between
> writing the file and reading it back through the import system, has
> caused people grief. When you're in a hole, the first thing to do is
> to *stop digging*.
>
> You can deprecate the heuristic if you want (and can figure out how),
> but a definite -1 on removing it without at least the usual
> deprecation period for backwards incompatible changes.
>

That part is easy: ImportWarning still exists so simply continuing to check
the directory and noticing when a difference exists that affects subsequent
imports and then raising the warning will handle that.


>
> It may also be worth tweaking the wording of the upgrade note in the
> What's New to mention the need to always invalidate the cache before
> scanning for new modules if you want to reliably pick up new modules
> created since the application started (at the moment the note really
> only mentions it as something to do after *creating* a new module).
>
>
As of right now with the check that's all that is needed, but yes, if the
deprecation does occur it would be worth changing it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130302/de0a1b0e/attachment.html>