[Distutils] Re: new import mechanism and a "small" distribution

[Greg Stein]

Just van Rossum wrote:
> 
> At 8:00 AM -0800 1/31/99, Greg Stein wrote:
> >...
> >Okay... enough background and rambling. If you're interested, go look at the
> >four messages in the thread titled "Freeze and new import architecture" in
> >the distutils-sig archive at:
> >http://www.python.org/pipermail/distutils-sig/1998-December/thread.html
> 
> I'm not on that sig so I missed your post originally. I agree with most you
> say here:
> http://www.python.org/pipermail/distutils-sig/1998-December/000077.html
> Especially that entries in sys.path should be loader instances (or
> directory paths).
> 
> Some questions:
> - what is the interface of a loader

By "loader", I will presume that you mean an instance of an Importer
subclass that is defining get_code(). Here is the method as defined by
imputil.Importer:

  def get_code(self, parent, modname, fqname):
    """Find and retrieve the code for the given module.

    parent specifies a parent module to define a context for importing.
It
    may be None, indicating no particular context for the search.

    modname specifies a single module (not dotted) within the parent.

    fqname specifies the fully-qualified module name. This is a
(potentially)
    dotted name from the "root" of the module namespace down to the
modname.
    If there is no parent, then modname==fqname.

    This method should return None, a 2-tuple, or a 3-tuple.

    * If the module was not found, then None should be returned.
    * The first item of the 2- or 3-tuple should be the integer 0 or 1,
      specifying whether the module that was found is a package or not.
    * The second item is the code object for the module (it will be
      executed within the new module's namespace).
    * If present, the third item is a dictionary of name/value pairs
that
      will be inserted into new module before the code object is
executed.
      This provided in case the module's code expects certain values
(such
      as where the module was found).
    """
    raise RuntimeError, "get_code not implemented"

That method is the sole interface used by Importer subclasses. To define
a custom import mechanism, you would just derive from imputil.Importer
and override that one method.

I'm not sure if that answers your question, however. Please let me know
if something is unclear so that I can correct the docstring.

Oh, geez. And I totally spaced on one feature of imputil.py. There is a
way to define an import mechanism for very simple uses. I created a
subclass named FuncImporter that delegates the get_code() method to a
user-supplied function. This allows a user to do something like this:

import imputil

def get_code(parent, modname, fqname):
  ...do something...

imputil.install_with(get_code)

The install_with() utility simply creates a FuncImporter with the
specified function and then installs the importer. No need to mess with
subclasses.

> - how are packages identified

If get_code() determines that the requested module is a package, then it
should return the integer 1 along with the code object for that
package's module. In the standard package system, the code object is
loaded from __init__.py.

An example: let's say that get_code() is called with (None, "mypkg",
"mypkg") for its arguments. get_code() finds "mypkg" in whatever path it
is configured for, and then determines that it represents a directory.
It looks in the directory for __init__.py or .pyc. If it finds it, then
mypkg is a real package. It loads code from __init__.py and returns (1,
code). The Importer superclass will create a new module for "mypkg" and
execute the code within it, and then label it as a package (for future
reference).

Internally, packages are labelled with a module-level name: __ispkg__.
That is set to 0 or 1 accordingly. The Importer that actually imports a
module places itself into the module with "__importer__ = self". This
latter variable is to allow Importers to only work on modules that they
imported themselves, and helps with identifying the context of an
importer (whether an import is being performed by a module within a
package, or not).

> - can it be in Python 1.6 or sooner?

I imagine that it could be in Python 1.6, but I doubt that it would go
into Python 1.5.2, as it has not had enough exposure yet. Guido's call
on all counts, though :-)

> PS: I was just wondering, why doesn't reload() use the standard import hook?

The standard import hook returns a *new* module object. reload() must
repopulate the module object.

I'm not sure that I answered your questions precisely, as they were
rather non-specific. If I haven't answered well enough, then please ask
again and I'll try again :-).

Cheers,
-g

--
Greg Stein, http://www.lyra.org/