[Import-SIG] PEP 420: Implicit Namespace Packages

[Nick Coghlan]

On Fri, Apr 20, 2012 at 6:18 AM, Eric V. Smith <eric at trueblade.com> wrote:
> This reflects (I hope!) the discussions at PyCon. My plan is to produce
> an implementation based on the importlib code, and then flush out pieces
> of the PEP.

This paragraph in the "Rationale" section is confusing:

"Namespace packages need to be installed in one of two ways: either
all portions of a namespace will be combined into a single directory
(and therefore a single entry in sys.path), or each portion will be
installed in its own directory (and each portion will have a distinct
sys.path entry)."

I would combine this with the following paragraph to make a single
cohesive explanation of the problem that needs to be solved:

"Namespace packages are designed to support being split across
multiple directories (and hence found via multiple sys.path entries).
In this configuration, it doesn't matter if multiple portions all
provide an __init__.py file, so long as each portion correctly
initialises the namespace package. However, Linux distribution vendors
(amongst others) prefer to combine the separate portions and install
them all into the *same* filesystem directory. This creates a
potential for conflict, as the portions are now attempting to provide
the *same* file on the target system - something that is not allowed
by many package managers. Allowing implicit namespace packages means
that the requirement to provide an __init__.py file can be dropped
completely, and affected portions can be installed into a common
directory or split across multiple directories as distributions see
fit."

> In particular, I want to make sure the PEP addresses the various
> objections that were raised, especially by Nick.

Yep. I'm happy with the conclusions we reached in the previous
discussion, but PEP 420 does need to describe them. Here's the gist of
it for the four points listed:

- for the first point, "practicality beats purity" pretty much carries
the day as far the Zen goes

- for the second point, the minor backwards compatibility risks are
acknowledged and accepted. My initial objection was based on a
misunderstanding of the consensus proposal. Once it was clarified that
the only "incompatibility" is that an import may now succeed where it
previously would have failed, I was no longer concerned. In contrast
to PEP 402, PEP 420 deliberately chooses to preserve consistent
behaviour of "import foo; import foo.bar" and "import foo.bar; import
foo", seeing that as being more important than preventing the
successful import of an empty (or otherwise non-package) subdirectory
of a sys.path location. This does mean some try/except import blocks
may need to updated to check the imported module or package for an
expected attribute or subpackage rather than just checking that the
import works, but has the major advantage of making the revised import
model much cleaner and easier to understand.

- the final two points will be addressed by having PEP 395 propose the
production of better *error messages* rather than introducing any
additional magic to the initialisation of sys.path[0] (see
http://mail.python.org/pipermail/import-sig/2012-March/000442.html).
The "are we in a package subdirectory?" heuristic mentioned in that
message will be based on this suggestion from Eric Snow:
http://mail.python.org/pipermail/import-sig/2012-March/000438.html

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia