[Import-SIG] New PEP draft: "Simplified Package Layout and Partitioning"

[Eric Snow]

On Wed, Jul 13, 2011 at 5:14 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 04:27 PM 7/13/2011 -0600, Eric Snow wrote:
> Outside of importlib and my sketch of a 2.x
> implementation for PEP 382, just how many meta importers *exist* in the
> outside world, after nearly nine years of PEP 302 being in existence?

So true.  I'm fine with taking the approach of  "handling sys.path
importers is good enough".

Perhaps one reason I have been pressing this is because of a project I
am working on that makes extensive use of meta importers.  And I
expect that everyone will be using it heavily within a few months of
its completion <wink>.

>> * sys.virtual_packages being a list vs. a dictionary
>
> Er, it's a set, not a list.  I'll change the bit that says that to highlight
> ``set()`` as a built-in type, vs. just the word "set".

Yeah, should have been set vs. dictionary.  But in the reality of how
meta importers factor in here, a dictionary it need not be.

>> And only one thing seems ambiguous when meta importers are left for
>> later.  If a module is loaded through a meta importer, which importer
>> handles a get_path() call?  When extend_virtual_paths is called, how
>> are meta-imported modules addressed?
>
> That's really up to the meta-importer.  You're really not supposed to use
> meta-importers to represent import *locations*; they're for extending or
> replacing import *policies*.  If you need locations, you make up a string to
> represent the location and put it in sys.path, after adding a path hook that
> recognizes the corresponding string.

That is a great explanation.    I guess that just makes me wonder what
part of the import process meta importers should respect.  Is it
anything goes?  The onus seems to be on the meta importer to make its
new import behavior as unsurprising as possible.  Regardless, this
doesn't have much bearing on this PEP past what you have already
addressed. :)

>> One last point:  This PEP results in two ways to provide a module for
>> a package (<NAME>.py in addition to <NAME>/__init__.py).  However, you
>> do offer a good distinction; __init__.py is for "self-contained"
>> packages.  Is it clear when to use which?  Will __init__.py go away
>> after a while?  Will we have to start looking in two places for a
>> package's code?
>
> I'll add something on that to the notes section:
>
> * While virtual packages are easy to set up and use, there is still
>  a time and place for using self-contained packages.  While it's not
>  strictly necessary, adding an ``__init__`` module to your
>  self-contained packages lets users of the package (and Python
>  itself) know that *all* of the package's code will be found in
>  that single subdirectory.  In addition, it lets you define
>  ``__all__``, expose a public API, provide a package-level docstring,
>  and do other things that make more sense for a self-contained
>  project than for a mere "namespace" package.

Sounds good.  Thanks for taking the time to clarify.

-eric