[Import-SIG] What if namespace imports weren't special?

[P.J. Eby]

I think one reason we're having trouble with naming and explaining 
this whole concept is that, really, the current Python import system 
is broken, compared to other languages.

In at least Perl, PHP, and Java, you don't have to do anything 
special to merge components in a single namespace from multiple parts 
of the class/include/autoload path.  We are thus having trouble 
trying to come up with a special name to describe these, when from a 
more objective perspective, what we are describing are "normal 
packages", with what Python has now being "restricted to a single 
directory packages".

It's for this reason that all packages being namespaces doesn't 
bother me for the term.  All packages *should* be namespace packages, 
pretty much.  It's the *non* namespaceyness of Python's default 
packages that's broken, not the term.  ;-)

If there really was a time machine, I like to think we'd go back and 
get Python's package import mechanism to just work this way from the 
outset (i.e. always combining shards across sys.path), and perhaps 
use the presence of .py[cod]/.so files as an indication of 
package-ness -- if indeed an indication is needed at all.

Actually...  here's an interesting idea.  Suppose that we define the 
rules so that any directory containing any file with an importable 
extension is a namespace package...  *but*, if one of those 
directories contains an __init__ module, that directory will be 
placed first on the package __path__.

See, the reason why dropping the need for __init__ was previously 
rejected was because it meant you could block the importing of a 
package later on the path.  *But*, if we always put the segment with 
__init__ first on the __path__, then any such blocking directories 
would not block the "real" package -- they'd just be accessible for imports.

If we did that, then there would be no need for any special flag 
files, and no need for special terminology.  The protocol in my draft 
would remain basically the same, except for moving the __init__ 
module's subpath to the front of __path__.  And instead of globbing 
for *.pypart or whatever, importers would just check whether there 
was a directory there at all.

The only backward compatibility that this can break is that you can 
import things you couldn't import before.  So, if you had a 
foo/bar.py, with 'foo' in a sys.path directory, and you also had a 
'foo' package, AND you relied on 'import foo.bar' raising an error, 
then it would no longer do so.  But, if you *had* a foo.bar module 
before, then under this scheme, 'import foo.bar' would still import 
the exact same file it did before, so nothing changes.

In other words, the first subdirectory with an __init__ gets to head 
up the new package's __path__, but ALL matching subdirectories will 
make up the tail.

The big advantage of this approach is that it doesn't require us to 
have a special name - it's just, "Enhanced Package Imports" or some 
such.  No special marker files to name, either.  Just, "hey, people 
want to put their package contents in more than one directory, and 
they don't always need an __init__.py."

Thoughts, anyone?