[Import-sig] deprecate ihooks?
M.-A. Lemburg
mal@lemburg.com
Fri, 04 Feb 2000 21:10:54 +0100
This is a multi-part message in MIME format.
--------------6726D2007785ACB228CD285A
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Just to throw some old 2 cents, I've attached some code I wrote
way back in 1997 on top of ihooks.py. It turns modules into
real classes with all the goodies of __getattr__ et al.
at no extra cost.
Perhaps this mechanism offers some new insights: by delegating
work to the objects in question (the modules) rather than
hooking together some meta objects... note that you can do
subclassing to add functionality to modules using this approach,
e.g. packages could be subclasses of a general package class, etc.
Anyway, just a thought you might want to consider... I'm too busy
right now to jump into this discussion again ;-)
--
Marc-Andre Lemburg
______________________________________________________________________
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/
--------------6726D2007785ACB228CD285A
Content-Type: text/python; charset=us-ascii;
name="ClassModules.py"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="ClassModules.py"
#!/usr/local/bin/python
""" Module-Class-Importer for Python (Version 0.6)
Modules in Python behave almost like classes,
but do not provide the same mechanisms, like inheritance,
baseclasses, special methods, etc.
This module provides an alternative module loader, that
is build on top of the ihooks.py-interface for the
builtin import statement. It works in a similar way,
the normal import does, but provides some extra features:
* when a module is requested, an instance of the
Module-class (or some subclass of Module) is created
and the actions 'find' and 'load' are redirected to
this instance via method calls
* after loading, a call to install_module copies all the
attributes from the "real" module object to the Module
instance (which costs some memory, but increases lookup speed),
thereby making it behave just like the original
* a reference to the original module object is kept,
so that 'from...import...' also works (since this statement
needs a real module object)
* whenever a module is referenced, the Module object is
returned, if possible, so even after having done 'from x import y'
at some point, 'import x' will return the Module object,
so hopefully all references to a module made by a Python
program should return the Module object, with all its
nice advantages (like catching AttributeErrors)
* Module provides a basic skeleton -- you can subclass
it and then give the ModuleClassImporter your class to use
(LazyModule is an example for this), if you don't like
some things, like copying attributes (e.g. use __getattr__
to redirect the lookup)
This module contains all necessary base classes (working ones,
not simply a bare framework), some Loaders, and
of course, the LazyModule which started this whole thing in
the first place.
For more information on how importing works, see ihooks.py
and ni.py.
----------------------------------------------------------------
Example of usage: Lazy Import for Python (see LazyImp.py)
---------------------------------------------------------------
History:
- 0.6: fixed for Python 1.5
Bugs:
- none, only unsupported features :-)
- I have tested it with Tkinter and a 10.000 line framework,
but of course... there may still be some imports out there,
I haven't taken into account yet.
(c) Marc-Andre Lemburg; all rights reserved
"""
__version__ = '0.6'
import sys,ihooks,imp,os
# so that it also works under Python 1.4
try:
__debug__
except:
__debug__ = 0
#
# A fast ModuleLoader
#
class FastModuleLoader(ihooks.ModuleLoader):
""" works like ModuleLoader, but uses imp's find_module, which makes
it somewhat faster
* note: file system hooks won't work here !!!
"""
def find_module(self,name,path=None):
m = self.find_builtin_module(name)
if m: return m
if path is None: path = sys.path
return imp.find_module(name,path)
#
# A preprocessing loader
#
# (parts taken from py_compile.py)
import marshal
def clong(x):
""" return the 4-byte long x as 4-byte string """
return chr(x&0xff)+chr((x>>8)&0xff)+chr((x>>16)&0xff)+chr((x>>24)&0xff)
class PreProcessingLoader(FastModuleLoader):
""" do some preprocessing when importing a module, that
has to be compiled first, i.e. is read in as source file
* leaves the rest to FastModuleLoader
"""
def load_module(self, name, stuff):
""" load the module name using stuff """
file, filename, (suff, mode, type) = stuff
# check if there already is a properly compiled version
pass
# if we have to handle a source file...
if type == imp.PY_SOURCE:
# read file
program = file.read()
# process program
program = self.preprocess(program)
# compile and try to write the .pyc-file (copied from py_compile.py)
code = compile(program, filename, 'exec')
codefilename = filename + (__debug__ and 'c' or 'o')
try:
fc = open(codefilename,'wb')
fc.write(imp.get_magic())
timestamp = long(os.stat(filename)[8])
fc.write(clong(timestamp))
marshal.dump(code,fc)
fc.close()
if os.name == 'mac':
import macfs
macfs.FSSpec(codefilename).SetCreatorType('Pyth', 'PYC ')
macfs.FSSpec(filename).SetCreatorType('Pyth', 'TEXT')
except IOError:
pass
else:
return FastModuleLoader.load_module(self, name, stuff)
# register and initialize module
m = self.hooks.add_module(name)
m.__file__ = filename
exec code in m.__dict__
return m
def preprocess(self,program):
""" do something with the code in program and return
the modified string
"""
program = "The_PreProcessingLoader_was_here = ':-)'\n" + program
return program
#
# The Module base class
#
class InternalVars: # container class
pass
class Module:
""" The module-works-as-a-class base class
* this class is instantiated for every new module loaded
by the SimulateImport mechanism
* you can subclass the class to add functionality and
pass the subclass to SimulateImport for it to be used
* important: local variables should always reside in
self.__moduleobj__, not in self directly (to avoid name
clashes)
* note: module initialization is done in the usual way, the
modules namespaces then copied to this object
* this class emulates the normal import-operation
"""
def __init__(self,name,loader,fromlist=None):
""" a module name is requested
* this method should NOT be overridden, instead override
startup() which is called, when this method finishes
"""
self.__moduleobj__ = m = InternalVars()
self.__name__ = name
m.loader = loader
m.fromlist = fromlist
m.found = 0
m.loaded = 0
m.modules = loader.modules_dict()
m.self = self
m.module = None # gets filled by load_module()
self.startup()
def startup(self):
""" module startup
* called when a module is requested
"""
self.find_module()
self.load_module()
def real_module(self):
""" return a real module object """
return self.__moduleobj__.module
def register(self):
""" makes an entry in modules pointing to this object
* loading a module through the loader normally also
registers the module, so a call to this method is
not needed
* note: if you want to do 'from..import..' with
this module later on, the registering MUST be done
by loader
"""
self.__moduleobj__.modules[self.__name__] = self
def find_module(self):
""" find the module """
m = self.__moduleobj__
m.stuff = m.loader.find_module(self.__name__)
if not m.stuff:
raise ImportError, 'Module: No module named %s'%name
m.found = 1
def load_module(self):
""" load the module and initialize it
* the module must already be found
* uses __moduleobj__.loader for loading
* calls .install_module to complete the job
"""
m = self.__moduleobj__
if m.loaded: return
if not m.found:
raise ImportError, 'Module: call %s.find_module() first'%self.__name__
else:
module = m.loader.load_module(self.__name__,m.stuff)
self.install_module(module)
m.loaded = 1
def install_module(self,module):
""" install the module in this objects namespace
* must be called after a module is loaded
"""
# keep a reference to the original
self.__moduleobj__.module = module
# copy all module attributes to this object
for k,v in module.__dict__.items():
setattr(self,k,v)
# create a reference in the real module object
setattr(module,'__moduleobj__',self.__moduleobj__)
def __repr__(self):
""" return some meaningful string describing self """
if self.__moduleobj__.loaded:
return "<%s '%s'>"%(self.__class__.__name__,self.__name__)
elif self.__moduleobj__.found:
return "<%s '%s', loading deferred>"%(self.__class__.__name__,self.__name__)
else:
return "<%s '%s', finding deferred>"%(self.__class__.__name__,self.__name__)
__str__ = __repr__
def __getattr__(self,x):
""" some unknown attribute is being requested """
raise AttributeError,'%s "%s" was looking for "%s"'%(self.__class__.__name__,self.__name__,x)
#
# The module-as-class importer
#
# ModuleImporter to be used:
ImporterBaseClass = ihooks.ModuleImporter
class ModuleClassImporter(ImporterBaseClass):
""" Module importer, that knows how to handle Module-objects correctly
"""
def __init__(self,module_class,*importer_class_init):
""" import modules by encapsulating them in an instance of
module_class
* modules_class must be a subclass of Module
* the other parameters are passed to the ImportClass
(see ihooks.py for details)
"""
apply(ImporterBaseClass.__init__,(self,)+importer_class_init)
self.module_class = module_class
def import_module(self, name, globals={}, locals={}, fromlist=None):
""" module import hook
"""
if self.modules.has_key(name): # fast path
m = self.modules[name]
# return the object, if possible
if fromlist is None:
#print 'Importer: import',name,'(found in sys.modules)',m
try:
return m.__moduleobj__.self
except:
return m
else:
# from..import.. insists on having the real thing !
#print 'Importer: from',name,'import',fromlist,'(found in sys.modules)',m
try:
return m.__moduleobj__.module
except:
return m
else:
if fromlist is None:
# normal 'import modulename'
#print 'Importer: import "%s" with %s'%(name,self.module_class.__name__)
module = apply(self.module_class,(name,self.loader,fromlist))
else:
# emulate 'from modulename import something'
# (note: this a hack... and not a nice one !)
#print 'Importer: from',name,'import',fromlist,'with',self.module_class.__name__
module = apply(self.module_class,(name,self.loader,fromlist))
# module has to be loaded for this to work
module.load_module()
module = module.real_module()
#print 'Importer: %s returned %s'%(self.module_class.__name__,module)
return module
--------------6726D2007785ACB228CD285A
Content-Type: text/python; charset=us-ascii;
name="LazyImp.py"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="LazyImp.py"
#!/usr/local/bin/python
"""
Lazy Import for Python (Version 0.6)
Loads modules only if they are needed and referenced.
This is done by overloading the builtin 'import'
statement, so no code changes are necessary.
Everything should work as normal, except that the
actual loading process is deferred until a module's
attribute is requested (you have to keep in mind,
that this can cause exceptions from the module
initialization process -- use lazyimp after debugging !)
* depends on the module ClassModules.py
*** Importing this module autoinstalls the Lazy Import Feature.
*** All subsequent imports will be done lazy.
*** If you don't like this, comment out the last line !
For more information, see the LazyModule-doc string below.
---------------------------------------------------------------
History:
- 0.6: fixed for Python 1.5
Bugs:
- none, only unsupported features :-)
- I have tested it with Tkinter and a 10.000 line framework,
but of course... there may still be some imports out there,
I haven't taken into account yet.
(c) Marc-Andre Lemburg; all rights reserved
"""
__version__ = '0.6'
import sys,ihooks,imp,os
from ClassModules import *
# base class to be used:
LazyModuleBaseClass = Module
class LazyModule(LazyModuleBaseClass):
""" Lazy Import for Python
Loads modules only if they are needed and referenced.
This is done by overloading the builtin 'import'
statement, so no code changes are necessary.
Everything should work as normal, except that the
actual loading process is deferred until a module's
attribute is requested (you have to keep in mind,
that this can cause exceptions from the module
initialization process -- use lazyimp after debugging !)
Hints:
- you can call the method load_module() of a lazy module
to force loading of the module (or simply reference
some attribute)
Caveats:
- attributes like __dict__ and __name__, that are provided
by the LazyImport-class, do not cause loading
- due to a Python internal limitation, from ... import ...
is not handled in a lazy fashion (wouldn't be too efficient anyway)
- debugging circular imports can become an even harder task
(uncomment the #print-statements to see what's going on)
"""
# finding the module is normally done when the object is created
# -- setting this to 1 defers finding too
__defer_find = 0
def startup(self):
""" lazy import module
"""
self.__moduleobj__.defer_find = self.__defer_find
if not self.__moduleobj__.defer_find:
self.find_module()
self.register()
def load_module(self,cause='*'):
""" do the actual import
* this can cause ImportErrors and raise exceptions, that
must be handled by the caller, i.e. the first reference
to a module might raise an exception !
* modules are only loaded once; any subsequent calls to this method
are silently ignored (i.e. ImportErrors are only raised
the first time, this method is used)
"""
if self.__moduleobj__.loaded: return
#print 'LazyModule: loading module "%s", looking for "%s" ...'%(self.__name__,cause)
if self.__moduleobj__.defer_find:
# find now
self.find_module()
# let the base class handle the rest
LazyModuleBaseClass.load_module(self)
#print 'LazyModule: module "%s" loaded'%self.__name__
def __getattr__(self,x):
""" the module's needed, so load it and return the
requested attribute afterwards
"""
#print self.__name__,'is looking for',x
if not self.__moduleobj__.loaded:
self.load_module(x)
return getattr(self,x)
else:
raise AttributeError,'%s "%s" was looking for "%s"'%(self.__class__.__name__,self.__name__,x)
def autoinstall():
""" install the Lazy Module Import feature """
mloader = FastModuleLoader()
mhandler = LazyModule
newimport = ModuleClassImporter(mhandler,mloader)
newimport.install()
#
# auto-install as new 'import' (comment out, if you don't like this)
#
autoinstall()
--------------6726D2007785ACB228CD285A--