[issue14604] spurious stat() calls in importlib

[Antoine Pitrou]

New submission from Antoine Pitrou <pitrou at free.fr>:

It seems importlib does multiple stat() calls on py files:

stat("/home/antoine/cpython/opt/Lib", {st_mode=S_IFDIR|0775, st_size=12288, ...}) = 0
stat("/home/antoine/cpython/opt/Lib/_sysconfigdata.py", {st_mode=S_IFREG|0664, st_size=16032, ...}) = 0
stat("/home/antoine/cpython/opt/Lib/_sysconfigdata.py", {st_mode=S_IFREG|0664, st_size=16032, ...}) = 0
open("/home/antoine/cpython/opt/Lib/__pycache__/_sysconfigdata.cpython-33.pyc", O_RDONLY) = 3


It also does multiple stat() calls on some directories:

stat("/home/antoine/cpython/opt/build/lib.linux-x86_64-3.3", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
stat("/home/antoine/.local/lib/python3.3/site-packages", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
stat("/home/antoine/.local/lib/python3.3/site-packages", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
stat("/home/antoine/.local/lib/python3.3/site-packages", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
open("/home/antoine/.local/lib/python3.3/site-packages", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3


That said, the number of system calls issued by 3.3 at startup is now much lower than with 3.2:

$ strace ./python -Sc pass 2>&1 | wc -l
512
$ strace python3.2 -Sc pass 2>&1 | wc -l
1018

----------
components: Library (Lib)
messages: 158546
nosy: brett.cannon, neologix, pitrou
priority: low
severity: normal
status: open
title: spurious stat() calls in importlib
type: performance
versions: Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14604>
_______________________________________