Import a module from a non-file?

[Alex Martelli]

Petri Savolainen wrote:

> I was trying to roll my own python code importer, but in the end, it seems
> that no matter what you try, it is always necessary to supply a REAL file
> object for python lower-level import machinery to work. Is this really
> true? Or did I miss something?

No, it's not true.  Read PEP 302, http://www.python.org/peps/pep-0302.html ,
for details.  Basically, you can write and install "Importer" objects,
which may return "Loader" objects (an importer an also be a loader, if
you wish, so importer.find_module can do a "return self" as long as the
importer object also has a load_module method).  A loader's method
load_module must return the fully loaded module object, and it can
built it in any way it pleases -- e.g. with new.module from standard
module new to make a new empty module object, then (if for example it
has obtained the Python source of the module from wherever) an exec
of that source using the new module's dictionary as locals and globals.

Let me give a trivial example made by overriding __import__ -- NOT the
recommended mechanism, just simpler to explain!

import __builtin__
bi = __builtin__.__import__

def __import__(name, *args):
    if name == 'foo': return makefoo()
    return bi(name, *args)
__builtin__.__import__ = __import__

import new
import sys
def makefoo():
    mod = new.module('foo')
    source = '''
print "foo is being imported"
def foo(): return "hi, foo here"
'''
    exec source in mod.__dict__
    return mod

import foo
print foo.foo()
import foo
print foo.foo()

running this emits:

[alex at lancelot ba]$ python i.py
foo is being imported
hi, foo here
foo is being imported
hi, foo here
[alex at lancelot ba]$

note that each import invokes makefoo AGAIN: if you want to use
the cache conveniently maintained for you in sys.modules['foo']
after the first time foo is imported, you have to do that 
explicitly should you choose to use this "override __import__"
old-but-still-there functionality (caching for the new and
improved mechanisms of PEP 302 is different -- again, see the
PEP itself for all details).


Alex