[Import-SIG] Namespace Packages resolution

["Martin v. Löwis"]

We were just discussing the namespaces PEPs here at PyCon.
We agreed to accept many, but not all principles of PEP 402;
PEP 382 was essentially rejected. As a consequence, Eric Smith
volunteered to write a new PEP covering the consensus.

Here are the basic principles:
- there will be two kinds of packages, "regular packages"
   and "namespace packages" (exact terminology subject to bikeshedding)
- "regular" packages have an __init__.py, and live in a single
   directory
- namespace packages can span multiple directories, and cannot
   have code on their own
- there will be no explicit marker for namespace packages;
   any directory on the path (sys.path or package __path__)
   can constitute a package. Package names equal directory
   names
- Importing a module/package keeps iterating over the parent
   path as before, and keeps with the current precedence:
   * if foo/__init__.py is found, a regular package is imported
   * if not, but foo.{py,pyc,so,pyd} is found, a module is
     imported
   * if not, but foo is found as a directory, it is recorded
   When search completes without importing a module, but it did
   find directories, then a namespace package is created. That
   namespace package
   * has an __name__ of the first directory that was found
   * has an __path__ which is the full list of directories
     that were collected

Of PEP 402, the following features where rejected:
- there is no support for code in a namespace package
   (i.e. you cannot use both foo/ and foo.py, but one
    will take precedence - depending on which one
    occurs first on the path)
- the PEP 402 terminology calling things "virtual"
   was rejected
- in the consensus spec, "import foo" alone will
   already trigger the path search, i.e. it is not deferred
   until a sub-level import occurs

A number of aspects are still undecided
- the API to dynamically update the __path__ was not
   immediately considered necessary. It may or may not be
   part of the PEP
- the specific API of PEP 302 finders and loaders for
   this feature was not specified; it will draw on the
   research done for PEP 382 and PEP 402.

The implementation will likely be provided once the import
mechanism in CPython is based on importlib.

Regards,
Martin