crap... missed the "s" in distutils. what fun now -- divergent threads (here and python-list). sigh. -----Original Message----- From: Greg Stein [mailto:gstein@lyra.org] Sent: Sunday, January 31, 1999 8:00 AM To: distutil-sig@python.org Cc: python-list@python.org Subject: new import mechanism and a "small" distribution A while back, Mark Hammond brought up a "new import architecture" proposal after some discussion with Jack Jansen and Guido. I responded in my usual diplomatic style and said "feh. the two-step import style is bunk." Then, I expounded on a few of the ideas that I had come up with after working with a lot of custom import stuff (for COM support, for sticking .pyc files into resources or structured-storage files, or for creating distribs with very few files). In essence, I maintained that importers should really be a single step find-and-load, and that Python would be best served by having a list of object instances, each performing this find-and-load step. Imagine, if you will, replacing sys.path with a list of "directory importers" where each importer checks a directory for the requested file. Next step in our hypothetical case is to add a new importer into sys.path that checks the Windows Registry for paths and looks there, or an importer that looks inside the resource fork of a Mac file. Heck, or use the typical example: have an importer that grabs a code object from over the web. Okay... enough background and rambling. If you're interested, go look at the four messages in the thread titled "Freeze and new import architecture" in the distutils-sig archive at: http://www.python.org/pipermail/distutils-sig/1998-December/thread.html What I'm posting here is two things: 1) imputil.py : this is a module that defines an import utility class (imputil.Importer). There are a few subclasses in here for different types of importers. More detail below. 2) a "small distribution" : the demo is for Win32, but it the basic mechanism is quite portable. In this small distro, you can place onto somebody's computer a mere 5 files and NO registry keys. Python starts up just fine and you can import all modules from the standard library. Basically, imputil.Importer's logic was derived from knee.py, which is an emulation of Python's algorithm. Importer defines one method that subclasses should override: get_code(). The doc strings and examples in the module cover that method relatively well. The Importer class will install an import hook and "chain" to the previous hook. This allows the use of multiple Importers, each one attempting to perform the import, and finally falling back to the builtin import mechanism. (note that there is no way to deinstall an importer at the moment, as it can't go "up" the chain to detach itself; the "right" way to do this is to define an Importer that contains a manageable list of Importers, and removing the chaining in favor of the manager iterating over the list) The small distribution replaces site.py and uses an archive of the standard library to fetch code objects from. This library is named "py15.pyl" in the demo, and is generated by the batch file in support/regen.bat (with the help of support/easygen.py). The resulting library is portable, as it is merely a giant glom of .pyc files and a marshalled table of contents (mapping module names to file positions). Look in site.py for the SimpleArchive class to get a feel for how to write a simple subclass of Importer. As another demo, imputil.py contains a class named DirectoryImporter. This subclass attempts to emulate Python's behavior for importing from a directory. It is instantiated with a directory to look in, for modules and packages. If a module is found, it will perform the usual .pyc caching if necessary (or use the cached .pyc). There is a demo function named imputil._test_dir() that will read sys.path and construct a chain of DirectoryImporter instances. You can then import away and Python will actually use the Importers rather than sys.path iteration. Lastly, imputil.py contains two (untested) subclasses for importing from a package contained in archive-type files (such as .gz, .zip, .tar, or a Mac file of resources). For example, you could distribute "package.gz" along with a gzip Importer. When somebody said "import package.module", it would look inside package.gz for the "module" module. I think that's it. This is the first release of this stuff, so there are bound to be improvements or bugfixes or just simple comments :-). Please feel free to speak up. thx, -g -- Greg Stein, http://www.lyra.org/