[Import-SIG] New PEP draft: "Simplified Package Layout and Partitioning"

[Greg Slodkowicz]

Dear all,
I've been working on a proof-of-concept implementation of PEP 402 as
part of my import-related Google Summer of Code project (the
repository is at https://bitbucket.org/jergosh/pep-402). I made the
changes to my ImportEngine-based import but the logic should be the
same as the original __import__ implemented in importlib. I haven't
studied the CPython implementation but I'm assuming it's similar.

I've ran into the following problem: prior to PEP-402, if we import
Foo.Bar.Baz, first Foo and then Foo.Bar are imported by recursion
finally, if these succeed, also Foo.Bar.Baz. Then __import__ returns
the root module, in this case Foo. But suppose Foo and Bar are virtual
packages, i. e. there is a file Foo/Bar/Baz.py somewhere on the path,
lacking any __init__.py files. As far as I understand the proposal,
then there would be no Foo to return.

One way to get around this would be to create an empty module on each
import level (Foo and Foo.Bar in this case) and set appropriate
__path__ on each of them. To my mind, this would require changing the
way get_subpath() works.

Currently, the requirement is that
"Each importer is checked for a get_subpath() method, and if present,
the method is called with the *full name* of the module/package the
path is being constructed for. The return value is either a string
representing a subdirectory for the requested package, or None if no
such subdirectory exists."

If instead we allow passing a 'partial' name (Foo and Foo.Bar) of a
virtual package to get_subpath() and indicate (by an extra parameter)
that we are looking for a purely virtual module, we could create
__path__ entries for each of these parent modules.

For that reason, in my implementation I introduced an extra parameter
' as_parent' to _gcd_import(), indicating that if a module or package
with a given name is not found, a subdirectory (e. g. Foo/Bar) on the
import path should be accepted as well and an analogical extra
parameter in get_subpath().

Hope this makes sense. I'd appreciate any feedback.

Best regards,
Greg