[Python-Dev] importlib

[Brett Cannon]

I have updated the benchmark to now measure importing source w/o writing
bytecode, importing source & writing bytecode, and importing bytecode w/
source (as I don't care about sourceless import performance).

Now, before you look at these numbers, realize that I have not once tried to
profile importlib to see where its hot spots are (only optimization I have
done is cut down on the stat calls since Python 3.1 when I re-developed the
ABCs). I'm sure if I profiled the code and wrote key bits in C these
performance numbers would improve reasonably quickly.

Anyway, on my 2.2 GHz MacBook, this leads to::

import.c
sys.modules [ 223337 223036 223362 ] best is 223362
Built-in module [ 23347 23319 23331 ] best is 23347
Bytecode w/ source [ 6624 6607 6608 ] best is 6624
Source w/o bytecode [ 4643 4674 4655 ] best is 4674
Source writing bytecode [ 2063 2145 2204 ] best is 2204

importlib
sys.modules [ 43423 43414 43426 ] best is 43426
Built-in module [ 9130 9115 9120 ] best is 9130
Bytecode w/ source [ 1554 1556 1556 ] best is 1556
Source w/o bytecode [ 1351 1351 1353 ] best is 1353
Source writing bytecode [ 786 843 810 ] best is 843

importlib / import.c:
sys.modules  19%
Built-in module  39%
Bytecode w/ source  23%
Source w/o bytecode  29%
Source writing bytecode  38%

What does this show? Stuff that requires a lot of I/O has the smallest
performance difference (source writing bytecode), but where there is as
little I/O as possible (bytecode w/ source) import.c wins as it has to do
less. This is also why sys.modules is so damn fast; it's the smallest amount
of C code you can run while importlib has standard Python calling overhead.

It should also be pointed out that importlib has fully implemented PEP 302
and intentionally has the loaders using their own exposed PEP 302 APIs. This
means there are a lot more methods calls than in the C version, along with a
lot less corners cut in the name of performance. So while importlib will be
slower simply because it's implemented in C, it will also be slower because
the darn thing is actually written to follow the PEPs we have along with
making it easier for people to subclass and benefit from the import code.

Anyway, as I have said, I need to hit 100% compatibility when running the
test suite -- run importlib.test.regrtest to see where it fails now; also
read that file as it has notes as to why the known failures are happening --
before I start worrying about bootstrapping and performance and that will
all be no sooner than Python 3.3.

On Thu, Jul 15, 2010 at 04:55, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Thu, Jul 15, 2010 at 4:06 PM, Brett Cannon <brett at python.org> wrote:
> >> In any case, here my results under a Linux system:
> >>
> >> $ ./python -m importlib.test.benchmark
> >> sys.modules [ 323782 326183 326667 ] best is 326667
> >> Built-in module [ 33600 33693 33610 ] best is 33693
> >>
> >> $ ./python -m importlib.test.benchmark -b
> >> sys.modules [ 1297640 1315366 1292283 ] best is 1315366
> >> Built-in module [ 58180 57708 58057 ] best is 58180
> >
> > And this is what might make evaluating importlib tough; while the
> > performance is 25% of what it is for import.c, being able to import
> > over 300,000 times/second is still damn fast.
>
> Yeah, I think the numbers where the filesystem gets involved are going
> to be more relevant. Modules that have already been cached and those
> built in to the executable aren't likely to dominate interpreter and
> application startup times (which is the main thing I'm worried about
> seeing regress).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100716/23be6935/attachment.html>