[Python-Dev] requirements for moving __import__ over to importlib?

[Antoine Pitrou]

On Tue, 7 Feb 2012 15:07:24 -0500
Brett Cannon <brett at python.org> wrote:
> 
> Now I'm going to be upfront and say I really did not want to have this
> performance conversation now as I have done *NO* profiling or analysis of
> the algorithms used in importlib in order to tune performance (e.g. the
> function that handles case-sensitivity, which is on the critical path for
> importing source code, has a platform check which could go away if I
> instead had platform-specific versions of the function that were assigned
> to a global variable at startup).

From a cursory look, I think you're gonna have to break (special-case)
some abstractions and have some inner loop coded in C for the common
cases.

That said, I think profiling and solving performance issues is critical
*before* integrating this work. It doesn't need to be done by you, but
the python-dev community shouldn't feel strong-armed to solve the issue.

> IOW I really do not look forward to someone saying "importlib is so much
> slower at importing a module containing ``pass``" when (a) that never
> happens, and (b) most programs do not spend their time importing but
> instead doing interesting work.

Well, import time is so important that the Mercurial developers have
written an on-demand import mechanism, to reduce the latency of
command-line operations.

But it's not only important for Mercurial and the like. Even if you're
developing a Web app, making imports slower will make restarts slower,
and development more tedious in the first place.

> So, if there is going to be some baseline performance target I need to hit
> to make people happy I would prefer to know what that (real-world)
> benchmark is and what the performance target is going to be on a non-debug
> build.

- No significant slowdown in startup time.

- Within 25% of current performance when importing, say, the "struct"
  module (Lib/struct.py) from bytecode.

Regards

Antoine.