[Python-Dev] Re: [Zope3-dev] Zip import and sys.path manipulation (was Re: directory hierarchy proposal)

[Guido van Rossum]

> (CC'd to python-dev because of possible impact on zipimport
> implementation/API)
> 
> At 08:00 AM 12/16/02 -0500, Jim Fulton wrote:
> >Each zope package __init__ file would have a bit of boilerplate code
> >that would include all zope directories in sys.path:
> >
> >   # Boilerplate code that combines all zope directories in sys.path
> >   import sys, os
> >   __path__ = [os.path.join(p, 'zope') for p in sys.path]
> >
> >Jim
> 
> That should probably read:
> 
> __path__ = [os.path.join(p, 'zope') for p in sys.path if 
> isinstance(p,StringTypes)]
> 
> or else it could break in 2.3 if somebody's putting non-strings on
> sys.path.  I don't remember exactly how the python-dev discussion
> turned out, but I kind of got the impression that it was going to be
> allowable in 2.3.

Correct.  While the standard setup will still have only strings, we'll
support non-strings on sys.path and some users may want to use this.

> Also, I'm pretty sure it will not give the expected/desired results
> in conjunction with zipfile imports.

I think it will, actually; if /path/foo.zip is a zip file on sys.path,
/path/foo.zip/blah references a directory inside that zip file.

> For these and related reasons, I would suggest that a slightly
> different snippet be used instead (the names are lousy, improvements
> welcome):
> 
> from namespace_packages import makeNamespacePath
> __path__ = makeNamespacePath('zope')
> 
> And then make whatever carries "namespace_packages" be something
> that's distributed separately from Zope, preferably part of the
> standard library or distutils (and backported where possible).  If
> we can make it so that the __init__ code never *has* to change, that
> will minimize the likelihood of incompatible __init__.py's floating
> around.  Meanwhile, a module that actually *knows* something about
> what can be on sys.path (and how to interpret it) can do so.

This is not a bad idea: put the logic to add stuff to a package's
__path__ according to a certain policy in the standard library.

(Of course, getting it into Python 2.2 would require another release,
2.2.3 -- which may happen anyway.)

> In order to support something like this, importer objects would need
> to be able to return something that's the moral equivalent of
> "os.path.join" on an existing sys.path entry; either another
> importer object or a string that can be handled by one.  At this
> point, checking for a '.zip' extension at the end of a string isn't
> going to cut it any more.

As I said above, I think it will.

Time's running short (we're close to releasing Python 2.3 alpha 1);
we should decide on a name and a specific policy ASAP.  I propose the
following:

  from pkgutil import extended_path
  __path__ = extended_path(__path__, __name__)

Where pkgutil.py could be something like this:

  import sys.path

  def extended_path(path, name):
      """Extend a package's path.  (XXX more.)"""
      path = path[:]
      for dir in sys.path:
          if isinstance(dir, (str, unicode)):
              dir = os.path.join(dir, name)
              if dir not in path and os.path.isdir(dir):
                  path.append(dir)
      return path

--Guido van Rossum (home page: http://www.python.org/~guido/)