[Python-ideas] My objections to implicit package directories

[Guido van Rossum]

On Mon, Mar 12, 2012 at 11:07 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> So far, I've only heard two *positive* statements in favour of
> implicit package directories:
>
> 1. Java/Perl/etc do it that way.
>
> I've already made it clear that I don't care about that argument. If
> it was all that compelling, we'd have implicit self by now. (However,
> clearly Guido favours it in this case, given his message that arrived
> while I was writing this one)

Honestly, I don't really care about "compatibility" with Java or Perl.
However that *both* of those languages do it this way (BTW what does
Ruby do?) is an argument that this is a *natural* or *intuitive* way
of setting things up. In fact, Python today also uses this: a package
P lives in a directory named P. Plain and simple. Users can
immediately understand this. Collapsing multiple directories named P
along sys.path is also pretty natural, given that we already have the
behavior of searching along sys.path. The requirement of having an
__init__.py file however is a wart.

> 2. It (arguably) makes it easier to convert an existing package into a
> namespace package
>
> With implicit package directories, you just delete your empty
> __init__.py file to turn an existing package into a namespace package.
> With a PEP 382 style directory suffix, you have to change your
> directory name to append the ".pyp" (and, optionally, delete your
> __init__.py file, since it's now going to be ignored anyway).
>
> Barry's also tried to convince me that ".pyp" directories are somehow
> harder for distributions to deal with, but his only example looked
> like trying to use "yield from" in Python 3.2 and then complaining
> when it didn't work.

I hope I've added some indication that it's also harder to deal with
in version control systems.

> However, so long as the backwards compatibility from PEP 402 is
> incorporated, and the new PEP proposed a specific addition to the
> tutorial to document the "never CD into a package, never double-click
> a file in a package to execute it, always use -m to execute modules
> from inside packages" guideline (and makes it clear that you may get
> strange and unpredictable behaviour if you ever break it), then I can
> learn to live with it. IDLE should also be updated to allow correct
> execution of submodules via F5 (I guess it will need some mechanism to
> be told what working directories to add to sys.path).

Those are all sensible requests.

> It still seems to me that moving to a marker *suffix* (rather than a
> marker file) as PEP 382 proposes brings all the real practical
> benefits of implicit package directories (i.e. no empty __init__.py
> files wasting space) and absolutely *none* of the pain (i.e. no
> backwards compatibility concerns, no ambiguity in the filesystem to
> module hierarchy mapping, still able to fix direct execution of
> modules inside packages rather than having to explain forevermore why
> it doesn't work), but Guido clearly feels otherwise.

I expect pain in different places.

-- 
--Guido van Rossum (python.org/~guido)