__file__ access extremely slow

Gabriel Genellina gagsl-py2 at yahoo.com.ar
Thu Jun 4 22:33:52 EDT 2009


En Thu, 04 Jun 2009 22:24:48 -0300, Zac Burns <zac256 at gmail.com> escribió:

> The section of code below, which simply gets the __file__ attribute of
> the imported modules, takes more than 1/3 of the total startup time.
> Given that many modules are complicated and even have dynamic
> population this figure seems very high to me. it would seem very high
> if one just considered the time it would take to load the pyc files
> off the disk vs... whatever happens when module.__file__ happens.

> Code: [fixed]
> ################################
> for module in sys.modules.itervalues():
>       try:
>               path = module.__file__
>       except (AttributeError, ImportError):
>               return
> ################################>

__file__ is just an instance attribute of module objects. Although a  
custom importer *might* define a special module type which *could* use a  
special computed attribute, I doubt so...
module.__file__ just returns a string, when it exists. Built-in modules  
have no __file__ attribute set, and some entries in sys.modules may be set  
to None. These should be the only exceptions.

> The calculation appears to be cached though, so a subsequent check
> does not take very long.
> From once python starts and loads the main module to after all the
> imports occur and this section executes takes 1.3sec. This section
> takes 0.5sec. Total module count is ~800.

Are you sure you posted the actual code?
That "return" statement would stop the iteration as soon as it hits a  
builtin module, or a None flag.

I'd say the time is spent somewhere else, or you're misinterpreting your  
results.
BTW, what's the point of all this?

-- 
Gabriel Genellina




More information about the Python-list mailing list