[Distutils] Is there any harm in clearing sys.path_importer_cache from time to time?

[Jim Fulton]

On Jun 23, 2006, at 11:53 AM, Phillip J. Eby wrote:

> At 11:37 AM 6/23/2006 -0400, Jim Fulton wrote:
>> I have a script based on setuptools that invokes easy install to
>> check for
>> and download newer distributions.  I allow users to specify  
>> whether or
>> not they want unzipped downloads.  As a result, easy_install will
>> sometimes
>> replace a zip file with a directory or the other way around.  This
>> leads to
>> problems in the invoking program because the sys.path_importer_cache
>> is stale.  Does anyone know if it is safe to clear the cache from
>> time to
>> time?  If so, I'd be inclined to clear it any time I invoke
>> easy_install.
>
> Hm.  easy_install should be doing this itself...  oh wait, it's  
> clearing the *zip directory cache*, which fixes the problem of  
> overwriting a zip with a zip.  You're having a problem when  
> overwriting a zip with a directory.  Now I understand why sometimes  
> I still get reports of behavior that resembles this problem, but  
> then they aren't reproducible.

Actually, I happened to be having a problem overriding a directory  
with a zip.  This was in a test in which I demonstrate being able to  
unzip and went back to not unzipping in a later example.  After the  
egg is reinstalled unzipped, the cache entry has None, left over from  
finding the directory before.

This was in a doctest which invoked my buildout script as a  
subprocess, which then invoked  easy_install as a subprocess.  When  
the buildout script started, the egg in question was a directory.  It  
then ran easy_install, which replaced the directory with a zip file.  
Then the buildout script tried to load the distribution for the  
installed file, which failed.

I haven't been able to manually find the right combination to make  
easy_install override a distribution manually, so I'm a bit puzzled.   
If you wish me to pursue that I will.


> I'll fix easy_install to also clear the path importer cache for a  
> path entry when it clears the zip directory cache for that entry.

OK. Note that that won't help me, because the cache I'm affected by  
is in the parent
process.

> (And to answer your general question, it's fine to clear the  
> path_importer_cache, per PEP 302's advice to do so whenever you add  
> new import hooks.  It just slows down the next import search a bit.)

Cool, cause that solves my immediate problem. :)

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org