[Python-ideas] Clear all caches

[Serhiy Storchaka]

On 01.04.15 17:37, Andrew Barnert wrote:
> I can see this being useful as just a way of standardizing an API for
> third-party modules that have a cache-clearing function. If there are
> a lot of them, having most of them do it the same way would make them
> easier to discover.

How will you discover them? Do you want to enumerate all loaded modules 
and all functions in these modules to check if they look as cache 
clearing functions? What about caches in class scope and dynamically 
created caches?


>> May be the behavior should be opposite: clear caches of specified
>> level and *lower*. Then by default only level 0 will be cleared and
>> users will be able to use higher levels for long-living caches.
>
> Higher vs. lower doesn't really matter that much; you can just define
> short, medium, and long as 0, -1, -2 instead of 0, 1, 2, right?

Right. But 0, 1, 2 can look less strange than 0, -1, -2. If all actually 
used values are negative, then we should revert the scale.

> Anyway, why do you want this? Is there some cache that's using too
> much memory in an app of yours? Or is it more about getting a
> semi-clean start on the interactive interpreter? Or something
> different?

This idea was inspired by a series of MemoryErrors on some buildbots. 
Some of these errors are provoked by overfilled caches (in particular 
linecache). I don't know how large other caches can be, but linecache 
can grow over 23 MB only on the stdlib and standard tests. See also 
issues #23838 and #23839.

http://bugs.python.org/issue23838
http://bugs.python.org/issue23839

> And which of the implicit caches all over the stdlib made you want
> this (since the stdlib was your motivating example)?

First of all it's linecache. But some cache can grow even more. For 
example re cache grows unlimitedly if you generates patterns dynamically 
from user data and use module level functions.

> Also, this seems like something that other platforms (special-purpose
> math languages, interactive SQL interpreters, etc.) might have. If
> so, have you looked around to see how they do it?

I don't see how this idea implemented in particular library can help to 
solve general problem without support in the stdlib.