__file__ access extremely slow

[Steven D'Aprano]

On Thu, 04 Jun 2009 18:24:48 -0700, Zac Burns wrote:

> The section of code below, which simply gets the __file__ attribute of
> the imported modules, takes more than 1/3 of the total startup time.

How do you know? What are you using to time it?


[...]
> From once python starts and loads the main module to after all the
> imports occur and this section executes takes 1.3sec. This section takes
> 0.5sec. Total module count is ~800.
> 
> Python version is 2.5.1
> 
> Code:
> ################################
> for module in sys.modules:
> 	try:
> 		path = module.__file__
> 	except (AttributeError, ImportError):
> 		return
> ################################


You corrected this to:

for module in sys.modules.itervalues():
       try:
               path = module.__file__
       except (AttributeError, ImportError):
               return

(1) You're not importing anything inside the try block. Why do you think 
ImportError could be raised?

(2) This will stop processing on the first object in sys.modules that 
doesn't have a __file__ attribute. Since these objects aren't 
*guaranteed* to be modules, this is a subtle bug waiting to bite you.




-- 
Steven