[Python-Dev] Embedded Python startup is slow
Stefan Behnel
stefan_ml at behnel.de
Thu Mar 24 19:03:39 CET 2011
bruce bushby, 24.03.2011 18:39:
> On Thu, Mar 24, 2011 at 5:05 PM, Stefan Behnel wrote:
>> bruce bushby, 24.03.2011 16:58:
>> On my desktop pc, when I run the most simple "Hello World" .... 78% of the
>>> overall execution time is spent opening files....most of which don't
>>> exist.
>>
>> How did you measure that?
>
> I used "strace -c hello.py"
Ok, I tried that as well, using a self-compiled CPython 3.2. On the first
run, "open()" was at the top, on the subsequent runs, "read()" was. So it
looks like a non-problem to me.
First run:
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00 0.000026 0 677 581 open
0.00 0.000000 0 145 read
0.00 0.000000 0 1 write
...
Second run:
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00 0.000016 0 145 read
0.00 0.000000 0 1 write
0.00 0.000000 0 677 581 open
...
The exact timings are likely bogus as the numbers are way too low to be
measured accurately. The higher number of calls is partly due to the ABI
version specific .so files that CPython 3.2 uses.
>> I'd expect that after the first startup, most of the relevant parts of the
>> file system are cached by the OS, so a subsequent run should be quick as all
>> file existence tests will run from memory. Sure, it's a bit of overhead to
>> call into the OS, and I don't know how expensive that is on ARM-Linux, but
>> given that you also claim it to be expensive on your desktop, I must say
>> that I'm a bit surprised about your above claim.
>
> On my pc, the "Hello World" script obviously executes very quickly....so
> trying to open 160 odd files that don't exist is negligible, but it still
> happens.
That brings us to the question why your processes are so short-running that
the startup time starts to matter. And, if they are so short-running, what
prevents the disk cache (or whatever kind of storage system you use) from
speeding up the open() requests. However, that's the right kind of question
to discuss on python-list, not python-devel.
Stefan
More information about the Python-Dev
mailing list