[Python-Dev] Issue #11051: system calls per import

[Scott Dial]

On 1/31/2011 1:38 PM, Brett Cannon wrote:
> I should mention that I have considered implementing a caching finder
> and loader for filesystems in importlib for people to optionally
> install to use for themselves. The real trick, though, is should it
> only cache hits, misses, or both? Regardless, though, it would be a
> very simple mixin or subclass to implement if there is demand for this
> sort of thing.

I have in the past implemented a PEP302 finder/loader zipfile-based
cache. On campus, I use a version of python installed to my home
directory that is on an NFS share. I found such a cache often gave
slower startup times for applications like bzr and hg.

My cache merely stores things it finds things in sys.path and loads from
the zipfile names that it knows and storing those that it doesn't. I
make no attempt to invalidate the cache contents once stored. So, I am
already talking about a best-case scenario for caching. I'm not sure how
you could invalidate the cache without paying the cost of all the normal
syscalls that we are trying to avoid.

My finder/loader is not bug-free, but I'd be glad to make it available
to someone if they want to play around with it.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu