[Import-SIG] PEP 382 update

[Nick Coghlan]

On Thu, Oct 20, 2011 at 12:16 PM, Barry Warsaw <barry at python.org> wrote:
> I vaguely recall that something similar has been discussed on the mailing list
> before, but that there were problems with directory name markers.  I could be
> misremembering, and will try to find details in my archives.

My recollection is similar, but I think the latest version may address
those objections by allowing the existing "package/__init__.py"
convention to still indicate a package directory *as well as* the new
"package.pyp" convention.

> Eric did remark last night that while PEP 402 is broader in scope, and *seems*
> useful, we really don't know what it will break.  Still, we need to get this
> feature moving again.

However, what if PEP 402 was *also* rewritten to only look at
directories named "package.pyp" rather than "package" when building
the virtual package paths? It still has a more coherent story for
handling namespace package initialisation than PEP 382, *without*
slowing down existing module and package imports.

In the latest PEP 382 update, the PEP proposes that finding a
package/__init__.py file *not stop the sys.path scan* (it's actually
inconsistent currently, but that behaviour is what the latest
additions describe). That means all imports of packages get slower
since the whole sys.path is always scanned in order to populate
__path__.

I believe the PEP 402 approach is much cleaner: both foo.py and
foo/__init__.py would stop the sys.path scan *immediately* (thus
eliminating any performance impact on existing imports), but
subpackage imports ("import foo.bar" or "from foo import bar") will
attempt to either convert an existing "foo" module into a package, or
else create a new package, by scanning the whole of sys.path for
"foo.pyp" directories. The "foo/__init__.py" approach would create
self-contained packages that are explicitly closed to extension.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia