[Import-SIG] Where to discuss PEP 382 vs. PEP 402 (namespace packages)?

[Nick Coghlan]

On Mon, Mar 12, 2012 at 11:43 AM, Barry Warsaw <barry at python.org> wrote:
> Just a quick note about PEP 382.  As it's currently written (i.e. with the
> .pyp extension on directories), I worry about whether it will be possible to
> support both Python 3.2 and Python >= 3.3 with a single vendor package.  This
> may only be an interim problem until we never care about Python 3.2 anymore 5
> years from now, or it might be solvable by backporting the PEP into the
> vendor's version of Python 3.2, or some other solution such as symlinks or
> what not (though I would definitely not want to reintroduce a huge symlink
> farm again).  Or maybe marker files such as described in the original version
> of the PEP are more appropriate after all.

Why is this a problem? PEP 382 still supports ordinary __init__.py
based package directories, so why can't the vendor packages just use
that and ignore PEP 382 altogether until they drop 3.2 support? None
of the existing namespace package mechanisms will break, the PEP just
introduces a new simpler alternative for code that doesn't need to
care about older versions of Python.

Delaying adoption of a new feature due to the need to maintain
compatibility with older versions of Python is hardly a novel
situation for a Linux distribution.

> Or maybe PEP 402 is the right approach after all, and Nick's concerns about
> its interactions with PEP 395 can be addressed in other ways.

I'd be interested in how - under PEP 402, it's completely impossible
to tell *from the filesystem alone* where a package hierarchy starts.
The information simply isn't there. Without some form of explicit
filesystem marker, there's always going to be a fundamental ambiguity
between "this is a sys.path directory with some top level modules" and
"this is a top-level package directory". Since fixing main module
imports requires the ability to unambiguously translate a filesystem
path into a sys.path entry and a module reference, the lack of
information is fatal to the primary goal of PEP 395.

> What I'd really like Guido to do is to provide his opinion about which PEP in
> general he favors.  Then I think we can hash out the details and corner cases
> here in this mailing list before a final pronouncement is made.

I thought he did that in the initial email (i.e. stating a general
preference for PEP 402). I just want to make it clear that accepting
PEP 402 means rejecting key parts of PEP 395 as a natural consequence,
as the PEP 402 approach to namespace packages eliminates currently
available information that PEP 395 will need in order to do its job.
After exploring the consequence of PEP 402 and the path entry vs
package directory ambiguity it introduces, I now believe package
layouts still fall under the general guideline that explicit is better
than implicit.

If anyone can demonstrate how to make all the main module invocation
methods that would *just work* under PEP 395 still work in a PEP 402
world, then I'll be happy to withdraw my objection, but I simply don't
see how it can be done. Under PEP 402, the precise meaning of a given
filesystem layout depends on the contents of sys.path. That means you
can't go back the other way and derive the "one true sys.path entry"
for that layout, because there will always be at least two valid
answers where there is currently only one.

I'm just repeating myself now, though, so I'll stop posting about it
until someone proposes a specific implementation strategy that would
resolve the incompatibility between the two PEPs. In the absence of
such a strategy, my position on the namespace package PEPs will
remain: "I want to implement PEP 395 to fix direct execution of
modules inside packages, which requires a 1:1 mapping between the
filesystem layout and the Python module hierarchy created by the
default import mechanism. I am therefore opposed to the implicit
packages in PEP 402 and in favour of the explicit packages in PEP
382".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia