<br><br><div class="gmail_quote">On Wed, Feb 8, 2012 at 20:26, Nick Coghlan <span dir="ltr"><<a href="mailto:ncoghlan@gmail.com">ncoghlan@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On Thu, Feb 9, 2012 at 2:09 AM, Antoine Pitrou <<a href="mailto:solipsis@pitrou.net">solipsis@pitrou.net</a>> wrote:<br>
> I guess my point was: why is there a function call in that case? The<br>
> "import" statement could look up sys.modules directly.<br>
> Or the built-in __import__ could still be written in C, and only defer<br>
> to importlib when the module isn't found in sys.modules.<br>
> Practicality beats purity.<br>
<br>
</div>I quite like the idea of having builtin __import__ be a *very* thin<br>
veneer around importlib that just does the "is this in sys.modules<br>
already so we can just return it from there?" checks and delegates<br>
other more complex cases to Python code in importlib.<br>
<br>
Poking around in importlib.__import__ [1] (as well as<br>
importlib._gcd_import), I'm thinking what we may want to do is break<br>
up the logic a bit so that there are multiple helper functions that a<br>
C version can call back into so that we can optimise certain simple<br>
code paths to not call back into Python at all, and others to only do<br>
so selectively.<br>
<br>
Step 1: separate out the "fromlist" processing from __import__ into a<br>
separate helper function<br>
<br>
def _process_fromlist(module, fromlist):<br>
# Perform any required imports as per existing code:<br>
# <a href="http://hg.python.org/cpython/file/aba513307f78/Lib/importlib/_bootstrap.py#l987" target="_blank">http://hg.python.org/cpython/file/aba513307f78/Lib/importlib/_bootstrap.py#l987</a><br>
<br></blockquote><div><br></div><div>Fine by me.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Step 2: separate out the relative import resolution from _gcd_import<br>
into a separate helper function.<br>
<br>
def _resolve_relative_name(name, package, level):<br>
assert hasattr(name, 'rpartition')<br>
assert hasattr(package, 'rpartition')<br>
assert level > 0<br>
name = # Recalculate as per the existing code:<br>
# <a href="http://hg.python.org/cpython/file/aba513307f78/Lib/importlib/_bootstrap.py#l889" target="_blank">http://hg.python.org/cpython/file/aba513307f78/Lib/importlib/_bootstrap.py#l889</a><br>
return name<br></blockquote><div><br></div><div>I was actually already thinking of exposing this as importlib.resolve_name() so breaking it out makes sense.</div><div><br></div><div>I also think it might be possible to expose a sort of importlib.find_module() that does nothing more than find the loader for a module (if available).</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Step 3: Implement builtin __import__ in C (pseudo-code below):<br>
<br>
def __import__(name, globals={}, locals={}, fromlist=[], level=0):<br>
if level > 0:<br>
name = importlib._resolve_relative_import(name)<br>
<div class="im"> try:<br>
module = sys.modules[name]<br>
except KeyError:<br>
</div> # Not cached yet, need to invoke the full import machinery<br>
# We already resolved any relative imports though, so<br>
# treat it as an absolute import<br>
return importlib.__import__(name, globals, locals, fromlist, 0)<br>
# Got a hit in the cache, see if there's any more work to do<br>
if not fromlist:<br>
# Duplicate relevant importlib.__import__ logic as C code<br>
# to find the right module to return from sys.modules<br>
elif hasattr(module, "__path__"):<br>
importlib._process_fromlist(module, fromlist)<br>
return module<br>
<br>
This would then be similar to the way main.c already works when it<br>
interacts with runpy - simple cases are handled directly in C, more<br>
complex cases get handed over to the Python module.<br></blockquote><div><br></div><div>I suspect that if people want the case where you load from bytecode is fast then this will have to expand beyond this to include C functions and/or classes which can be used as accelerators; while this accelerates the common case of sys.modules, this (probably) won't make Antoine happy enough for importing a small module from bytecode (importing large modules like decimal are already fast enough).</div>
</div>