[Import-SIG] What if namespace imports weren't special?

[Nick Coghlan]

On Mon, Jul 11, 2011 at 12:34 PM, P.J. Eby <pje at telecommunity.com> wrote:
> The big advantage of this approach is that it doesn't require us to have a
> special name - it's just, "Enhanced Package Imports" or some such.  No
> special marker files to name, either.  Just, "hey, people want to put their
> package contents in more than one directory, and they don't always need an
> __init__.py."

It does mean that the pkgutil changes to handle sys.path extensions
will need to scan sys.modules looking for packages (i.e. modules with
__path__ attributes) rather than the more limited subset that would
have been stored in sys.partitioned_packages (although not adding
extra global state is actually a win in my book).

Removing the need for __init__.py as a package marker would also
eliminate quite a lot of newbie confusion when it comes to using
packages.

However, I think the explicitly partitioned package approach is going
to be an easier sell, as it's *obvious* that it won't break existing
code. While examples of existing code that will break under a
"partitioned by default" model are going to be hypothetical and
contrived, they're also pretty easy to come up with.

There's also a performance impact on app startup time - currently most
package imports stop as soon as they hit a matching directory. Under a
"partitioned by default" scheme, all package imports (including things
like "logging" and "email" which currently get a hit in the first zip
file for the standard library) would have to scan the entirety of
sys.path just in case there are additional shards lying around. For
large applications, that additional overhead is going to add up.

So I don't think implicit partitioning is really going to fly at this
point. That said, I wouldn't oppose tweaking the partitioned package
design to eventually support dropping the requirement for explicit
".pyp(art)" files (i.e. by always placing the directory with
__init__.py at the head of the list).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia