<div class="gmail_quote">On Thu, Feb 9, 2012 at 5:34 PM, Robert Kern <span dir="ltr"><<a href="mailto:robert.kern@gmail.com">robert.kern@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On 2/9/12 10:15 PM, Antoine Pitrou wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Thu, 9 Feb 2012 17:00:04 -0500<br>
PJ Eby<<a href="mailto:pje@telecommunity.com" target="_blank">pje@telecommunity.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Thu, Feb 9, 2012 at 2:53 PM, Mike Meyer<<a href="mailto:mwm@mired.org" target="_blank">mwm@mired.org</a>> wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
For those of you not watching -ideas, or ignoring the "Python TIOBE<br>
-3%" discussion, this would seem to be relevant to any discussion of<br>
reworking the import mechanism:<br>
<br>
<a href="http://mail.scipy.org/pipermail/numpy-discussion/2012-January/059801.html" target="_blank">http://mail.scipy.org/<u></u>pipermail/numpy-discussion/<u></u>2012-January/059801.html</a><br>
<br>
Interesting. This gives me an idea for a way to cut stat calls per<br>
</blockquote>
sys.path entry per import by roughly 4x, at the cost of a one-time<br>
directory read per sys.path entry.<br>
</blockquote>
<br>
Why do you even think this is a problem with "stat calls"?<br>
</blockquote>
<br></div>
All he said is that reading about that problem and its solution gave him an idea about dealing with stat call overhead. The cost of stat calls has demonstrated itself to be a significant problem in other, more typical contexts.</blockquote>
<div><br></div><div>Right. It was the part of the post that mentioned that all they sped up was knowing which directory the files were in, not the actual loading of bytecode. The thought then occurred to me that this could perhaps be applied to normal importing, as a zipimport-style speedup. (The zipimport module caches each zipfile directory it finds on sys.path, so failed import lookups are extremely fast.)</div>
<div><br></div><div>It occurs to me, too, that applying the caching trick to *only* the stdlib directories would still be a win as soon as you have between four and eight site-packages (or user specific site-packages) imports in an application, so it might be worth applying unconditionally to system-defined stdlib (non-site) directories.</div>
<div><br></div></div>