[Import-SIG] PEP 395 (Module aliasing) and the namespace PEPs

[Eric Snow]

On Tue, Nov 15, 2011 at 11:29 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> One of the fixes PEP 395 (module aliasing) proposes is to make running
> modules inside packages by filename work correctly (i.e. without
> breaking relative imports and without getting the directory where the
> module lives directly on sys.path which can lead to unexpected name
> clashes). The PEP currently states [1] that this can be made to work
> with both PEP 382 and PEP 402
>
> In current Python, fixing this just involves checking for a colocated
> __init__.py file. If we find one, then we work our way up the
> directory hierarchy until we find a directory without an __init__.py
> file, put *that* on sys.path, then (effectively) rewrite the command
> line as if the -m switch had been used.
>
> The extension to the current version of PEP 382 is clear - we just
> accept both an __init__.py file and a .pyp extension as indicating
> "this is part of a Python package", but otherwise the walk back up the
> filesystem hierarchy to decide which directory to add to sys.path
> remains unchanged.
>
> However, I'm no longer convinced that this concept can actually be
> made to work in the context of PEP 402:
>
> 1. We can't use sys.path, since we're trying to figure out which
> directory we want to *add* to sys.path
> 2. We can't use "contains a Python module", since PEP 402 allows
> directories inside packages that only contain subpackages (only the
> leaf directories are required to contain valid Python modules), so we
> don't know the significance of an empty directory without already
> knowing what is on sys.path!
>
> So, without a clear answer to the question of "from module X, inside
> package (or package portion) Y, find the nearest parent directory that
> should be placed on sys.path" in a PEP 402 based world, I'm switching
> to supporting PEP 382 as my preferred approach to namespace packages.
> In this case, I think "explicit is better than implicit" means, "given
> only a filesystem hierarchy, you should be able to figure out the
> Python package hierarchy it contains". Only explicit markers (either
> files or extensions) let you do that - with PEP 402, the filesystem
> doesn't contain enough information to figure it out, you need to also
> know the contents of sys.path.

Ouch.  What about the following options?

Indicator for the top-level package?  No
Leverage __pycache__?  No

Merge in the idea from PEP 382 of special directory names?  To borrow
an example from PEP 3147:

alpha.pyp/
    one.py
    two.py
    beta.py
    beta.pyp/
        three.py
        four.py

So package directories are explicitly marked but PEP 402 otherwise
continues as-is.  I'll have to double-check, but I don't think we
tried this angle already.

-eric


>
> Regards,
> Nick.
>
> [1] http://www.python.org/dev/peps/pep-0395/#fixing-direct-execution-inside-packages
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Import-SIG mailing list
> Import-SIG at python.org
> http://mail.python.org/mailman/listinfo/import-sig
>