Barry Warsaw wrote:
Thanks everybody for providing great input on this aspect of the PEP. I've updated the open issues section to include a list of the possible resolutions for bytecode-only imports. Unless anybody has more ideas, it might just be time to get a BDFL pronouncement.
I think the benchmarking in the bytecode-only section is still too weak. "evidence shows that the extra stats can be fairly costly to start up time" isn't a valid justification for breaking working code. Doing 4 stat calls instead of 5 on a directory miss just doesn't excite me very much without some genuine benchmarks across different operating systems and filesystems showing that reducing the number of stat calls by at best 20% will result in a measurable reduction in import times for real modules (where we can expect the import time to be dominated by the execution of the actual module code rather than the time needed to find that code in the first place). Using the sample numbers Robert Collins posted: # Startup time for bzr (cold cache): $ drop-caches $ time bzr --no-plugins revno 5061 real 0m8.875s user 0m0.210s sys 0m0.140s # Hot cache $ time bzr --no-plugins revno 5061 real 0m0.307s user 0m0.250s sys 0m0.040s $ strace -c bzr --no-plugins revno 5061 % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 56.34 0.040000 76 527 read 28.98 0.020573 9 2273 1905 open 14.43 0.010248 14 734 625 stat 0.15 0.000107 0 533 fstat hot cache: % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 45.10 0.000368 92 4 getdents 19.49 0.000159 0 527 read 16.91 0.000138 1 163 munmap 10.05 0.000082 2 54 mprotect 8.46 0.000069 0 2273 1905 open 0.00 0.000000 0 8 write 0.00 0.000000 0 367 close 0.00 0.000000 0 734 625 stat Assuming all those stat errors are misses from the import system, we're looking at reducing that 625 figure down to 500: 125 fewer failed calls. With a hot cache, the impact is too small for strace to even measure. With a cold cache, it is 1.75 milliseconds: only 1.25% of the system time consumed in the script's execution, and not even registering relative to the 9 second wall clock time. Without significant measurable performance gains, a mere aesthetic preference isn't enough to justify inflicting subtle breakage on even a small subset of our users. Even aside from the issue of a lack of benchmarks to justify the breakage, bytecode only imports *cannot* legitimately be broken without at least one release where they generate Deprecation Warnings. Cheers, Nick. P.S. I actually started this thread as a +0 to the idea of dropping bytecode only imports. Over the course of the discussion I've shifted to a firm -1 in the absence of some proper comparative benchmarks to justify the change in semantics. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------