Packages & __path__ (was:Re: [Python-Dev] Python 2.3a1 release -- Dec 31)

[Just van Rossum]

Samuele Pedroni wrote:

> Basically we want the same thing, but so either
> 
> 1) __path__ manipulations are supported 2) or __path__ is deprecated
> and we get wide-importing by default (that means the union of all
> relevant "directories" is considered and not just one)
> 
> indeed my use-case for 1) is to get 2).
> 
> Both solutions are OK, and yes the 2nd simplifies some things.

Hey, a consensus ;-)

Here's a concrete proposal:

  - deprecate pkg.__path__ *manipulation*, starting with Python 2.3a1.
  - pkg.__path__ will be set as before.
  - if len(pkg.__path__) > 1:
        issue DeprecationWarning
        search submodules on pkg.__path__, the old way.
    else:
        search submodules on sys.path, the new way.

(Fancier: keep a copy of __path__ in a separate variable, say
__orgpath__, issue a warning and invoke the old logic if __orgpath__ !=
__path__. This is robust even against alterations of __path__[0], but
seems a little overkill to me.)

This should be fully b/w compatible.

> [ Then supporting path/to/a.zip/pkg1 in sys.path only would be still
> a feature and would be easier to support or just Jython could want to
> support it. ]

Or: if Python ever grows a virtual file system, this feature could be
readded. I'd prefer to leave it out for now.

> My only remaining worry, once the above is sorted out, is whether the
> PEP302 scheme can be in the future expanded to support ihooks like
> functionalities (like those used by Quixote) in a non-messy way.

Right now there are two ways to go about addressing the issue:

A) turn/wrap the builtin Path importer into a real object on
sys.meta_path.

B) create a subset of ihooks.py as a new module (written in Python),
reimplementing some of the path import logic.

Either one should provide an API for adding file type handlers. (I'm not
so sure ihooks even provides a straightforward way for multiple
independent packages to each add their own file type handlers. For
example I don't see how Quixote could currently be compatible with other
packages doing similar things, short of chaining __import__ hooks. A new
solution should have it as a major goal to make this possible.)

B could be a prototype for A, so it seems logical to initially go for B.
This doesn't (IMO) need to be in first alpha, so there's more time to
experiment and discuss.

Just