[Import-SIG] Where to discuss PEP 382 vs. PEP 402 (namespace packages)?

[Nick Coghlan]

On Mon, Mar 12, 2012 at 9:02 AM, Guido van Rossum <guido at python.org> wrote:
> Martin has asked me to decide on PEP 382 vs. PEP 402 (namespace
> packages) in time for inclusion of the decision in Python 3.3.  As
> people who attended the language-sig know, I am leaning towards PEP
> 402 but I admit that at this point I don't have enough information. If
> I have questions, should I be asking them on the import-sig or on
> python-dev?

I agree with the others that import-sig is the right place.

> Is it tolerable if I ask questions even if the answer is
> somewhere in the archives? (I spent a lot of time reviewing the
> "pitchfork thread",
> http://mail.python.org/pipermail/python-dev/2006-April/064400.html,
> but that wasn't particularly fruitful, so I'm worried I'd just waste
> my time browsing the archives -- if the PEP authors did their jobs
> well the PEPs should include summaries of the discussion anyways.)

When PJE first proposed PEP 402 I was a fan, but I changed my mind
once I realised it was fundamentally incompatible with PEP 395 (where
I want to fix the way we initialise sys.path[0] so that direct
execution of modules inside packages isn't broken).

With the status quo or PEP 382's explicit namespace packages, the
interpreter can look at the *filesystem* to figure out where the root
directory of the package lives, as the explicit package markers (i.e.
__init__.py files or *.pyp extensions) mean there is an unambiguous
1:1 mapping from a filesystem path to a (sys.path entry, module
reference) pair.

With PEP 402, however, the filesystem layout becomes ambigous - you
*can't* derive the appropriate sys.path entry from the filesystem any
more, because the meaning of the filesystem layout *depends on* what
you put in sys.path.

To give a simple example:

Status quo:

  myproject/
      __init__.py
      __main__.py
      module.py
      tests/
          __init__.py
          test_module.py

Clearly, "myproject" is a Python package with "myproject.module" and
"myproject.tests.test_module" as the contents. Only the parent
directory of "myproject" should be placed on sys.path. The package can
be executed as a script via "python -m myproject".

PEP 382:

  myproject.pyp/
      __main__.py
      module.py
      tests.pyp/
          test_module.py

No real change from the status quo - the "__init__.py" marker files
are simply replaced by the "*.pyp" marker extension.

PEP 402:

  myproject/
      __main__.py
      module.py
      tests/
          test_module.py

Uh-oh, now we have a problem. This is either an executable directory
(designed to be run as "python myproject" and providing a top-level
"module" and "tests.test_module") or an executable package (designed
to be run as "python -m myproject" and providing "myproject.module"
and "myproject.tests.test_module"). Because the filesystem layout is
ambiguous, there's no way for the interpreter to figure out what the
developer meant and we'd be stuck with the current broken* sys.path[0]
initialisation forever.

I've described the implications of the two namespace PEPs briefly in
PEP 395, which also has links to the relevant import-sig threads:
http://www.python.org/dev/peps/pep-0395/#compatibility-with-pep-382
http://www.python.org/dev/peps/pep-0395/#incompatibility-with-pep-402

*PEP 395 also explains my rationale for calling our current
sys.path[0] initialisation mechanism broken:
http://www.python.org/dev/peps/pep-0395/#why-are-my-imports-broken

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia