[Python-Dev] Re: [Zope3-dev] Zip import and sys.path manipulation (was Re: directory hierarchy proposal)

[Guido van Rossum]

> Minor nit: .pth semantics should be identical to site.py .pth
> semantics.  I.e. lines that start with "import" should be exec'd.

Barry & I had an offline discussion about this.

Some conclusions:

- pth files for sys.path and pth-ish files for packages should
  probably have different extensions; otherwise e.g. paths named in
  zope.pth would be added both to sys.path and to zope.__path__, which
  is a bad idea.  I propose using pkg as the extension for package.s

- Nobody remembers why there is a special case in site.py for lines in
  the .pth file that start with 'import', except that Martin von
  Loewis once wanted this.  I propose not to duplicate this
  functionality in .pkg files.

- We need to look both for subdirectories and for .pkg files --
  different styles of administering a system call for both approaches.

- We should only accept subdirectories that have an __init__.py.

- I'm punting for now on an extension to cope with finding things in
  other importers.

Current draft, incorporating all these changes:

######################################################################
"""Utilities to support packages."""

import os
import sys

def extend_path(path, name):
    """Extend a package's path.

    Intended use is to place the following code in a package's __init__.py:

        from pkgutil import extend_path
        __path__ = extend_path(__path__, __name__)

    This will add to the package's __path__ all subdirectories of
    directories on sys.path named after the package.  This is useful
    if one wants to distribute different parts of a single logical
    package as multiple directories.

    It also looks for *.pkg files beginning with __name__.  This
    feature is similar to *.pth files, except it doesn't special-case
    lines starting with 'import'.
    """
    pname = os.path.join(*name.split('.')) # Reconstitute as relative path
    sname = os.extsep.join(name.split('.')) # Just in case os.extsep != '.'
    sname_pkg = sname + os.extsep + "pkg"
    init_py = "__init__" + os.extsep + "py"
    path = path[:]
    for dir in sys.path:
        if isinstance(dir, (str, unicode)) and os.path.isdir(dir):
            subdir = os.path.join(dir, pname)
            # XXX This may still add duplicate entries to path on
            # case-insensitive filesystems
            initfile = os.path.join(subdir, init_py)
            if subdir not in path and os.path.isfile(initfile):
                path.append(subdir)
            # XXX Is this the right thing for subpackages like
            # zope.app?  It looks for a file named "zope.app.pkg"
            pkgfile = os.path.join(dir, sname_pkg)
            if os.path.isfile(pkgfile):
                try:
                    f = open(pkgfile)
                except IOError, msg:
                    sys.stderr.write("Can't open %s: %s\n" %
                                     (pkgfile, msg))
                else:
                    for line in f:
                        line = line.rstrip('\n')
                        if not line or line.startswith('#'):
                            continue
                        path.append(line) # Don't check for existence!
                    f.close()
    return path
######################################################################

--Guido van Rossum (home page: http://www.python.org/~guido/)